Papers with cross-lingual transfer
Zero-Shot Cross-Lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on improving cross-lingual transferability of NMT model is under-explored. |
| Approach: | They propose a model that leverages a multilingual pretrained encoder to improve cross-lingual transferability. |
| Outcome: | The proposed model outperforms mBART and m2m-100 on a zero-shot cross-lingual transfer task. |
Can Monolingual Pretrained Models Help Cross-Lingual Classification? (2020.aacl-main)
Copied to clipboard
| Challenge: | Multilingual pretrained language models have shown impressive results for cross-lingual transfer, but due to the constant model capacity, multilingual pre-training usually lags behind the monolingual competitors. |
| Approach: | They propose to transfer the knowledge from monolingual pretrained models to multilingual ones to improve zero-shot cross-lingual classification by using machine translation systems. |
| Outcome: | The proposed methods outperform vanilla multilingual fine-tuning on two cross-lingual classification benchmarks. |
Meta-Learning a Cross-lingual Manifold for Semantic Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Recent work has found success with machine translation or zero-shot methods . however, these approaches can struggle to model how native speakers ask questions . |
| Approach: | They propose a meta-learning algorithm to leverage minimal annotated examples in new languages for few-shot cross-lingual semantic parsing. |
| Outcome: | The proposed approach trains a parser with maximum sample efficiency in six languages on ATIS. |
Fortification of Neural Morphological Segmentation Models for Polysynthetic Minimal-Resource Languages (N18-1)
Copied to clipboard
| Challenge: | Morphological segmentation for polysynthetic languages is challenging because of limited training data. |
| Approach: | They propose two new multi-task training approaches that improve performance for Mexican polysynthetic languages . they also propose cross-lingual transfer as a third way to fortify their neural model . |
| Outcome: | The proposed models improve on Mexicanero, Nahuatl, Wixarika and Yorem Nokki . the proposed models reduce the amount of parameters by close to 75% . |
Modality Matching Matters: Calibrating Language Distances for Cross-Lingual Transfer in URIEL+ (2026.eacl-srw)
Copied to clipboard
York Hay Ng, Aditya Khan, Xiang Lu, Matteo Salloum, Michael Zhou, Phuong Hanh Hoang, A. Seza Doğruöz, En-Shiun Annie Lee
| Challenge: | Existing linguistic knowledge bases such as URIEL+ lack a principled method for aggregating these signals into a single, comprehensive score. |
| Approach: | They propose a framework for type-matched language distances that unifies these signals into a robust, task-agnostic composite distance. |
| Outcome: | The proposed representations improve transfer performance when the distance type is relevant to the task, while yielding gains in most tasks. |
Improving Low-Resource Cross-lingual Parsing with Expected Statistic Regularization (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual syntactic analysis have been shown to be effective for low-resource languages. |
| Approach: | They propose to use low-order statistical functions to shape model distributions for semi-supervised learning on low-resource datasets. |
| Outcome: | The proposed method improves POS and LAS on 5 target languages and provides significant gains over strong cross-lingual-transfer-plus-fine-tuning baselines for modest amounts of label data. |
Cross-lingual Transfer of Reward Models in Multilingual Alignment (2025.naacl-short)
Copied to clipboard
| Challenge: | Recent studies in reward modeling schemes are skewed towards English, limiting the applicability of RLHF in multilingual alignments. |
| Approach: | They investigate cross-lingual transfer of English RMs by representation shifts . they also analyze cross-linguistic transfer of RM through the representation shift . |
| Outcome: | The results show that English RMs can be transferred across languages by 34% . |
FAD-X: Fusing Adapters for Cross-lingual Transfer to Low-Resource Languages (2022.aacl-short)
Copied to clipboard
| Challenge: | Adapter-based tuning is a technique that selectively updates language-specific parameters to adapt to a new language, rather than fine-tuning all shared weights. |
| Approach: | They propose to add light-weight adapters to multilingual pretrained language models (mPLMs) and add language-specific parameters to adapt to a new language. |
| Outcome: | The proposed adapter can enhance cross-lingual transfer from pretrained adapters for well-known named entity recognition and classification benchmarks. |
Investigating the Potential of Task Arithmetic for Cross-Lingual Transfer (2024.eacl-short)
Copied to clipboard
| Challenge: | Massively multilingual Transformer-based models (MMTs) can learn representations which have a degree of cross-lingual alignment despite being trained using purely unsupervised objectives. |
| Approach: | They propose a modular approach to cross-lingual transfer using task arithmetic . they show that modularity can be achieved even with full model fine-tuning . |
| Outcome: | The proposed approach shows strong performance on multilingual benchmarks encompassing both high-resource and low-resourced languages. |
Towards Making the Most of Cross-Lingual Transfer for Zero-Shot Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing unsupervised neural machine translation systems can degrade when labeled data is limited. |
| Approach: | They propose a multilingual pretraining and multilingual fine-tuning for facilitating cross-lingual transfer in zero-shot translation using a parallel dataset. |
| Outcome: | The proposed model outperforms state-of-the-art models on many-to-English translation by over 7.2 and 5.0 BLEU. |
From Monolingual to Multilingual FAQ Assistant using Multilingual Co-training (D19-61)
Copied to clipboard
Mayur Patidar, Surabhi Kumari, Manasi Patwardhan, Shirish Karande, Puneet Agarwal, Lovekesh Vig, Gautam Shroff
| Challenge: | Recent research on cross-lingual transfer shows state-of-the-art results on benchmark datasets using pre-trained language representation models like BERT. |
| Approach: | They propose a method to augment an annotated dataset with machine translations in target languages and fine-tune the PLRM jointly. |
| Outcome: | The proposed approach provides consistent gains on multiple benchmark datasets while requiring a single model for multiple languages. |
Cross-Lingual Transfer with Target Language-Ready Task Adapters (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for (zero-shot) cross-lingual transfer employ separate language and task adapters which can be arbitrarily combined to perform any task to any target language. |
| Approach: | They propose to fine-tune 'target language-ready' adapters to the target language to achieve better transfer performance without sacrificing the modularity of MAD-X. |
| Outcome: | The proposed adapters outperform MAD-X and BAD-X on most tasks and languages while maintaining the modularity of MAD. |
Unknown Script: Impact of Script on Cross-Lingual Transfer (2024.naacl-srw)
Copied to clipboard
| Challenge: | Existing models for high-resource languages are not available for all languages, and the vast majority of the world's languages are excluded from these models. |
| Approach: | They propose to use pre-trained models to analyze the effect of the target language and its script on cross-lingual transfer. |
| Outcome: | The proposed model is based on six models pre-trained on NER and POS tasks in the original script and romanized version. |
Massively Multilingual Transfer for NER (P19-1)
Copied to clipboard
| Challenge: | Existing approaches for cross-lingual transfer use a single source language, but there are exceptions. |
| Approach: | They propose two techniques for modulating the transfer, suitable for zero-shot or few-shot learning, respectively. |
| Outcome: | The proposed methods are much more effective than baseline models and rival oracle selection of the single best individual model. |
ContrastiveMix: Overcoming Code-Mixing Dilemma in Cross-Lingual Transfer for Information Retrieval (2024.naacl-short)
Copied to clipboard
| Challenge: | Multilingual pretrained language models have been widely adopted in cross-lingual transfer . however, training mPLMs on code-mixed data is counterproductive . |
| Approach: | They propose a method that disentangles contrastive loss between conflicting objectives . they propose mPLMs that train on code-mixed data for information retrieval . |
| Outcome: | The proposed model outperforms mDPR in cross-lingual transfer tasks . it is effective in sentence-level and token-level zero-shot tasks despite lack of target language data . |
ANHALTEN: Cross-Lingual Transfer for German Token-Level Reference-Free Hallucination Detection (2024.acl-srw)
Copied to clipboard
| Challenge: | ANHALTEN is a new evaluation dataset that extends the English hallucination detection dataset to German. |
| Approach: | They propose a dataset that extends the English hallucination detection dataset to German . they show that larger context length leads to better halluciation detection in german . |
| Outcome: | ANHALTEN is the first evaluation dataset that extends the English hallucination detection dataset to German. |
Inducing Language-Agnostic Multilingual Representations (2021.starsem-1)
Copied to clipboard
| Challenge: | Cross-lingual representations have the potential to make NLP techniques available to the vast majority of languages in the world, but they currently require large pretraining corpora or access to typologically similar languages. |
| Approach: | They propose to remove language identity signals from multilingual embeddings by re-aligning vector spaces of target languages to a pivot source language and removing language-specific means and variances. |
| Outcome: | The proposed approaches reduce cross-lingual transfer gap by 8.9 points (m-BERT) and 18.2 points (XLM-R) on average across all tasks and languages. |
Beyond the English Web: Zero-Shot Cross-Lingual and Lightweight Monolingual Classification of Registers (2021.eacl-srw)
Copied to clipboard
Liina Repo, Valtteri Skantsi, Samuel Rönnqvist, Saara Hellström, Miika Oinonen, Anna Salmela, Douglas Biber, Jesse Egbert, Sampo Pyysalo, Veronika Laippala
| Challenge: | Existing studies on register classification for web documents have limited results due to skewed datasets and low performance. |
| Approach: | They propose two new register-annotated corpora for French and Swedish . they show that deep pre-trained language models perform strongly in these languages . |
| Outcome: | The proposed models outperform existing models in English and Finnish and can match or surpass existing models. |
Meta-XNLG: A Meta-Learning Approach Based on Language Clustering for Zero-Shot Cross-Lingual Transfer and Generation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to learn shareable structures from low-resource languages are limited in the zero-shot setting. |
| Approach: | They propose a meta-learning framework to learn shareable structures from typologically diverse languages based on meta- learning and language clustering. |
| Outcome: | The proposed framework is able to learn shareable structures from typologically diverse languages with limited annotated data. |
CharSpan: Utilizing Lexical Similarity to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages (2024.eacl-short)
Copied to clipboard
| Challenge: | Existing models for ELRLs lack parallel corpora and monolingual corporata . authors propose novel character-span noise argumentation model to facilitate cross-lingual transfer . |
| Approach: | They propose a character-span noise argumentation model to facilitate cross-lingual transfer . they use character-size noise argumentations to regularize training data of HRL . |
| Outcome: | The proposed model outperforms baselines on closely related HRL-ELRL pairs from three different language families. |
Translation Errors Significantly Impact Low-Resource Languages in Cross-Lingual Learning (2024.eacl-short)
Copied to clipboard
| Challenge: | XNLI benchmarks use parallel versions of English evaluation sets in multiple target languages . a recent study found that translation errors exist in some low-resource languages resulting in incorrect estimates of cross-lingual transfer . |
| Approach: | They propose to measure the gap in performance between zero-shot evaluations on human-translated and machine-transcribed target text across multiple target languages. |
| Outcome: | The proposed benchmarks show that translation errors exist for Hindi and Urdu . the results corroborate previous studies that found translation errors in Hindi and urdu despite translation errors. |
DiTTO: A Feature Representation Imitation Approach for Improving Cross-Lingual Transfer (2023.eacl-main)
Copied to clipboard
| Challenge: | Zero-shot cross-lingual transfer has been shown to be sub-optimal across low-resource languages due to the skew in resource distribution in languages. |
| Approach: | They propose to jointly reduce feature incongruity between the source and target language and increase generalization capabilities of pre-trained multilingual transformers. |
| Outcome: | Empirical results show that the proposed approach outperforms the standard zero-shot fine-tuning method on multiple datasets across all languages using only unlabeled instances in the target language. |
Translation or Recitation? Calibrating Evaluation Scores for Machine Translation of Extremely Low-Resource Languages (2026.acl-short)
Copied to clipboard
| Challenge: | Existing studies show that performance across low-resource settings is variable, resulting in a significant barrier for the MT community. |
| Approach: | They propose to use FRED Difficulty Metrics to contextualize reported performance across different language pairs to determine whether breakthroughs reported in other contexts are artifacts of benchmark collection. |
| Outcome: | The proposed metrics explain a significant portion of result variability rather than model capability. |
Sharing, Teaching and Aligning: Knowledgeable Transfer Learning for Cross-Lingual Machine Reading Comprehension (2023.findings-emnlp)
Copied to clipboard
| Challenge: | X-STA is a new approach for cross-lingual machine reading comprehension . the variation of answer span positions in different languages makes it difficult to transfer knowledge across languages. |
| Approach: | They propose a method that leverages an attentive teacher to subtly transfer the answer spans of the source language to the answer output space of the target. |
| Outcome: | The proposed method outperforms state-of-the-art approaches on three multi-lingual datasets. |
Language-Aware Multilingual Machine Translation with Self-Supervised Learning (2023.findings-eacl)
Copied to clipboard
| Challenge: | Multilingual machine translation (MMT) is a challenging multitask optimization problem because of lack of a framework to learn language-specific parameters. |
| Approach: | They propose a self-supervised learning task that denies monolingual data to MMT . they then propose 'intra-distillation' task that co-trains with MMT task . |
| Outcome: | The proposed approach outperforms three state-of-the-art methods on 8-language and 15-language benchmarks. |
Zero-shot Cross-lingual Transfer With Learned Projections Using Unlabeled Target-Language Data (2023.acl-short)
Copied to clipboard
| Challenge: | Zero-shot cross-lingual transfer is enabled by pairing the language adapter in the target language with an appropriate task adapter within a source language. |
| Approach: | They propose to use unlabeled text to enhance zero-shot transfer by pairing language adapters with task adapters in a target language. |
| Outcome: | The proposed framework improves on three cross-lingual tasks with up to 11% relative improvement in Named Entity Recognition (NER), Question Answering (QA) and Natural Language Inference (NLI). |
Multi-view Subword Regularization (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing subword regularization methods for multilingual pretrained representations are suboptimal for multi-lingual transfer. |
| Approach: | They propose a method that enforces consistency between standard and probabilistic segmentations. |
| Outcome: | The proposed method improves the effectiveness of cross-lingual transfer by 2.5 points over standard methods. |
Multilingual Chart-based Constituency Parse Extraction from Pre-trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for extracting complete (binary) parses from pre-trained language models are expensive and time-consuming. |
| Approach: | They propose a chart-based method and an effective top-K ensemble technique to extractbinary parses from PLMs. |
| Outcome: | The proposed method can induce non-trivial parses for sentences from nine languages in an integrated and language-agnostic manner, and is robust to cross-lingual transfer. |
Analysis of Multi-Source Language Training in Cross-Lingual Transfer (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual transfer (XLT) methods address data scarcity problem . cross-linguistic transfer (xLT) techniques are effective at fine-tuning multilingual LMs . |
| Approach: | They propose to use multiple source languages to improve XLT by fine-tuning multilingual models . they propose to employ arbitrary combinations of source languages for XL to improve performance . |
| Outcome: | The proposed technique improves performance on language-agnostic or task-specific features by using multiple source languages. |
Distilling Multilingual Transformers into CNNs for Scalable Intent Classification (2022.emnlp-industry)
Copied to clipboard
| Challenge: | Existing multilingual models for voice assistants are limited by their prohibitive inference time and limited performance. |
| Approach: | They propose to distill and deploy multilingual Transformer models for voice assistants using a teacher-student framework that uses teacher-trained models to supervise student model training. |
| Outcome: | The proposed model outperforms a teacher model trained on unlabelled data and achieves equivalent performance. |
The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants (2024.acl-long)
Copied to clipboard
Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, Madian Khabsa
| Challenge: | Existing benchmarks for text comprehension only cover 30 languages, but lack of labeled data is a major obstacle to building functional systems in most languages. |
| Approach: | They present a multiple-choice machine reading comprehension dataset spanning 122 languages . they use it to evaluate the capabilities of multilingual masked language models and large language models . |
| Outcome: | The proposed dataset enables the evaluation of text models in high-, medium- and low-resource languages. |
AlignFreeze: Navigating the Impact of Realignment on the Layers of Multilingual Models Across Diverse Languages (2025.naacl-short)
Copied to clipboard
| Challenge: | Realignment techniques are often employed to enhance cross-lingual transfer in multilingual language models, but can degrade performance in languages that differ significantly from the fine-tuned source language. |
| Approach: | They propose a method that freezes either the lower half or upper half of the layers during realignment to prevent performance degradation. |
| Outcome: | The proposed method improves Part-of-Speech (PoS) tagging performance in languages where realignment fails. |
Cross-Lingual Transfer of Cognitive Processing Complexity (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies indicate that multilingual language models utilize structural similarities between languages to facilitate cross-lingual transfer. |
| Approach: | They propose a multilingual model that uses structural similarities between languages to facilitate cross-lingual transfer by a meaningful bias towards sentence length and cross-linguistic differences. |
| Outcome: | The proposed model can predict varied patterns for 13 languages, despite being fine-tuned only on English data. |
Improving Cross-lingual Transfer through Subtree-aware Word Reordering (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that multilingual language models are not effective when dealing with less-represented languages. |
| Approach: | They propose a powerful reordering method that learns word-order patterns conditioned on the syntactic context from a small amount of annotated data. |
| Outcome: | The proposed method outperforms baselines on a variety of tasks and is effective in both zero-shot and few-shot scenarios. |
Conversational Question Answering in Low Resource Scenarios: A Dataset and Case Study for Basque (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing datasets for conversational question answering systems are expensive and limited in resources . a dataset of thousands of dialogues and tens of thousands question answering turns is available for free . |
| Approach: | They aim to test the performance of Conversational Question Answering systems in non-English languages . they use a dataset built on top of Wikipedia sections about popular people and organizations . |
| Outcome: | The results show that the system can handle low-resource conditions comparable to English . the results also show that dialogue history models are not directly transferable to another language . |
Machine Translation for Livonian: Catering to 20 Speakers (2022.acl-short)
Copied to clipboard
| Challenge: | Livonian is one of the most endangered languages in Europe with just a tiny handful of speakers and virtually no publicly available corpora. |
| Approach: | They aim to develop machine translation between Livonian and English using a linguistic similarity test and a dataset of parallel and monolingual data. |
| Outcome: | The proposed systems and the collected data, including a manually translated and verified translation benchmark, are publicly released via OPUS and Huggingface repositories. |
Learning Compact Metrics for MT (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in machine translation and multilingual text generation have led researchers to adopt trained metrics such as COMET or BLEURT, which treat evaluation as a regression problem and use representations from multilingual pre-trained models such as XLM-RoBERTa or mBERT. |
| Approach: | They propose to use multilingual model capacity to improve model performance by transferring knowledge from one teacher to multiple students trained on related languages. |
| Outcome: | The proposed model yields 10.5% improvement over vanilla fine-tuning and reaches 92.6% of RemBERT’s performance using only a third of its parameters. |
Leveraging Multi-lingual Positive Instances in Contrastive Learning to Improve Sentence Embedding (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent trends in learning monolingual and multilingual sentence embeddings are based on contrastive learning (CL) among an anchor, one positive and multiple negative instances. |
| Approach: | They propose to leverage multiple positives to improve learning of multilingual sentence embeddings by using an anchor, one positive, and multiple negative instances. |
| Outcome: | The proposed approach improves retrieval, semantic similarity, and classification performance on unseen languages. |
MEDs for PETs: Multilingual Euphemism Disambiguation for Potentially Euphemistic Terms (2024.findings-eacl)
Copied to clipboard
Patrick Lee, Alain Chirino Trujillo, Diana Cuevas Plancarte, Olumide Ojo, Xinyi Liu, Iyanuoluwa Shode, Yuan Zhao, Anna Feldman, Jing Peng
| Challenge: | Euphemisms are a linguistic device used to soften or neutralize language that may otherwise be harsh or awkward to state directly. |
| Approach: | They train a multilingual transformer model to disambiguate potentially euphemistic terms in multilingual and cross-lingual settings. |
| Outcome: | The proposed model performs better than monolingual models on the disambiguation task compared to monolingual ones in multilingual and cross-lingual settings. |
Cross-lingual Few-Shot Learning on Unseen Languages (2022.aacl-main)
Copied to clipboard
| Challenge: | Large pre-trained language models have demonstrated the ability to obtain good performance on downstream tasks with limited examples in resource-rich languages. |
| Approach: | They propose to use a downstream sentiment analysis task to analyze the effectiveness of several few-shot learning strategies across 12 languages, including 8 unseen languages, to compare results. |
| Outcome: | The proposed model, XLM-R, gives the best performance on a task with few examples in resource-rich languages. |
Cross-Lingual Transfer from Related Languages: Treating Low-Resource Maltese as Multilingual Code-Switching (2024.eacl-long)
Copied to clipboard
| Challenge: | Multilingual models exhibit impressive cross-lingual transfer capabilities on unseen languages, but performance is impacted when there is a script disparity with the languages used in the model’s pre-training data. |
| Approach: | They propose a novel method to align a resource-rich language's script with a target language and train a classifier that can make informed decisions regarding the appropriate processing of each token. |
| Outcome: | The proposed model can be used to transfer a language's scripts across multiple languages, but it is suboptimal for mixed languages, where only a subset benefits while the rest is impeded. |
Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer (2026.acl-srw)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced natural language processing, yet their benefits remain concentrated in English and a small number of high-resource languages. |
| Approach: | They fine-tuned large language models (4B–671B parameters) on Arabic and evaluated zero-shot reading comprehension on Semitic languages and non-Semitic controls. |
| Outcome: | The results show that models with weak baselines improve across all languages, whereas strong-baseline models show only marginal gains regardless of language family. |
Only for the Unseen Languages, Say the Llamas: On the Efficacy of Language Adapters for Cross-lingual Transfer in English-centric LLMs (2025.acl-srw)
Copied to clipboard
| Challenge: | Most state-of-the-art large language models (LLMs) are trained mainly on English data, limiting their effectiveness on non-English, especially low-resource, languages. |
| Approach: | They train language adapters for 13 languages and evaluate their effectiveness on downstream tasks using either task adapters or in-context learning. |
| Outcome: | The proposed language adapters improve performance for languages not seen during pretraining, but provide negligible benefit for seen languages. |
Beyond Literal Token Overlap: Token Alignability for Multilinguality (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing studies have shown that token overlap is a strong predictor of multilinguality and cross-lingual knowledge transfer between languages with different scripts. |
| Approach: | They propose a subword token alignability metric to understand the impact and quality of multilingual tokenisation. |
| Outcome: | The proposed metric predicts multilinguality much better when scripts are disparate and the overlap of literal tokens is low. |
A Shared Geometry of Difficulty in Multilingual Language Models (2026.acl-short)
Copied to clipboard
| Challenge: | Large language models encode problem difficulty as an internal signal that can be linearly decoded from their residuals. |
| Approach: | They train linear probes on the AMC subset of the Easy2Hard benchmark, translated into 21 languages, and found difficulty-related signals emerge at two distinct stages of the model internals. |
| Outcome: | The results show that difficulty-related signals emerge at two distinct stages of the model internals, corresponding to shallow (early-layers) and deep (later-layer) representations, that exhibit functionally different behaviors. |
Comprehensive Study of Bilingual and Multi-category Instruction Pre-training (2026.findings-eacl)
Copied to clipboard
| Challenge: | Instruction pre-training (IPT) has recently gained attention as an intermediate stage between pre- and post-training for large language models. |
| Approach: | They study the optimal balance between raw and instruction-response data, languages, and task categories in an LLM instruction-respondence dataset. |
| Outcome: | The proposed model improves on English-centric and bilingual models using bilingual instruction-response datasets. |
Neural Cross-Lingual Event Detection with Minimal Parallel Resources (D19-1)
Copied to clipboard
| Challenge: | Existing methods for event detection (ED) rely on high-performance machine translation systems or manually aligned documents to achieve a decent performance. |
| Approach: | They propose a method that uses context-dependent translation to construct a lexical mapping between different languages and a shared syntactic order event detector for multilingual co-training. |
| Outcome: | The proposed method performs cross-lingual transfer and tackles the extremely annotation-poor scenario. |
Cross-lingual Argumentation Mining: Machine Translation (and a bit of Projection) is All You Need! (C18-1)
Copied to clipboard
| Challenge: | Argumentation mining (AM) requires the identification of complex discourse structures . existing resources are not adequate for assessing cross-lingual AM due to their heterogeneity or lack of complexity. |
| Approach: | They propose to use a dataset to translate persuasive student essays into German, French, Spanish, and Chinese to compare arguments mining and annotation projection. |
| Outcome: | The proposed methods perform better when using expensive human or cheap machine translations and almost eliminate loss from cross-lingual transfer. |
XQA-DST: Multi-Domain and Multi-Lingual Dialogue State Tracking (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for capturing dialogue data are expensive and limited in their application. |
| Approach: | They propose a domain-agnostic extractive question answering approach with shared weights across domains to disentangle complex domain information in ToDs. |
| Outcome: | The proposed model can efficiently leverage domain-agnostic QA datasets while being domain-scalable and open vocabulary in DST. |
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT (D19-1)
Copied to clipboard
| Challenge: | Pretrained contextual representation models have pushed forward the state-of-the-art on many NLP tasks. |
| Approach: | They propose to use a model that is pretrained on 104 languages for cross-lingual transfer. |
| Outcome: | The proposed model performs well on 5 NLP tasks covering 39 languages from various language families. |
Low-Rank Adaptation for Multilingual Summarization: An Empirical Study (2024.findings-naacl)
Copied to clipboard
| Challenge: | Pre-trained Large Language Models have significantly advanced NLP, but their ever-increasing size poses significant challenges for conventional fine-tuning. |
| Approach: | They investigate the potential of Low-Rank Adaptation (LoRA) in multilingual summarization, a task that is challenging and relatively unexplored. |
| Outcome: | The proposed method outperforms full fine-tuning and cross-lingual transfer strategies in multilingual summarization tasks. |
Towards more equitable question answering systems: How much more data do you need? (2021.acl-short)
Copied to clipboard
| Challenge: | Question answering datasets in English are relatively new, but lack of linguistic diversity in the field is a challenge. |
| Approach: | They propose to use translation and cross-lingual transfer to produce QA systems in multiple languages to improve their performance. |
| Outcome: | The proposed approaches take advantage of existing resources to produce QA systems in multiple languages. |
Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval (2022.coling-1)
Copied to clipboard
| Challenge: | State-of-the-art neural rankers are notoriously data-hungry and rarely used in multilingual and cross-lingual retrieval settings. |
| Approach: | They propose to use Sparse Fine-Tuning Masks and Adapters to transfer rankers trained on English data to other languages and cross-lingual setups by means of multilingual encoders. |
| Outcome: | The proposed methods outperform standard zero-shot transfer with full MMT fine-tuning while being more modular and reducing training times. |
Pushing the Limits of Low-Resource Morphological Inflection (D19-1)
Copied to clipboard
| Challenge: | Recent advances in morphological inflection generation have limited resources . antonisa and colleagues present a battery of improvements to improve performance under low-resource conditions . |
| Approach: | They propose a two-step attention architecture for the inflection decoder that uses two-segments attention and a multi-single-syllabic attention architecture. |
| Outcome: | The proposed model outperforms the state-of-the-art in low-resource languages by 15 percentage points . the proposed model also shows that it can be used to model monolingual data hallucinations . |
Domain Mismatch Doesn’t Always Prevent Cross-lingual Transfer Learning (2022.lrec-1)
Copied to clipboard
| Challenge: | Recent studies have reported that domain mismatch prevents cross-lingual transfer . UBLI and UNMT do not work well when underlying monolingual corpora come from different domains . |
| Approach: | They show that a simple initialization regimen can overcome domain mismatch in cross-lingual transfer . they pre-train word embeddings on concatenated domain-mismatched corpora and use them as initializations . |
| Outcome: | The initialization regimen can overcome the domain mismatch effect in cross-lingual transfer learning . the initializations were used for MUSE UBLI, UN Parallel UNMT, and the SemEval 2017 task . |
Predicting Foreign Language Usage from English-Only Social Media Posts (N18-2)
Copied to clipboard
| Challenge: | Social media is known for its multi-cultural and multilingual interactions, a natural product of which is code-mixing. |
| Approach: | They analyze 6 million tweets produced by 27 thousand multilingual users speaking 12 other languages besides English to build predictive models to infer non-English languages users speak exclusively from their tweets. |
| Outcome: | The proposed models are based on a corpus of 6 million tweets produced by 27 thousand multilingual users speaking 12 other languages besides English . they show that content, style and syntax are the most predictive of non-English languages that users speak on Twitter. |
Tomato, Tomahto, Tomate: Do Multilingual Language Models Understand Based on Subword-Level Semantic Concepts? (2025.findings-naacl)
Copied to clipboard
| Challenge: | a recent study shows that human understanding of text depends on general semantic concepts of words that are robust to their superficial forms. |
| Approach: | They evaluate the accuracy of multilingual multilingual language models based on subword-level semantics . they form "semantic tokens" by merging semantically similar subwords and embeddings based upon the results . |
| Outcome: | The proposed models are able to make predictions on multilingual tasks with different tokenizers and model sizes. |
DeFT-X: Denoised Sparse Fine-Tuning for Zero-Shot Cross-Lingual Transfer (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior studies have explored multiple approaches to combine task knowledge from task-specific data in a (high-resource) source language with language knowledge from unlabeled text in 'low-resourced' target language. |
| Approach: | They propose a composable sparse fine-tuning approach that learns task-specific and language-specific sparsen masks to select a subset of the pretrained model's parameters. |
| Outcome: | The proposed approach performs at par or outperforms SFT and other prominent cross-lingual transfer baselines. |
mOthello: When Do Cross-Lingual Representation Alignment and Cross-Lingual Transfer Emerge in Multilingual Models? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Pretrained multilingual models exhibit cross-lingual transfer ability, which is often attributed to a learned language-neutral representation during pretraining. |
| Approach: | They propose a synthetic task, Multilingual Othello, as a testbed to investigate the factors that contribute to the learning of a language-neutral representation. |
| Outcome: | The proposed approach induces the learning of language-neutral representation and facilitates cross-lingual transfer. |
Fine-Grained Analysis of Cross-Linguistic Syntactic Divergences (2020.acl-main)
Copied to clipboard
Dmitry Nikolaev, Ofir Arviv, Taelin Karidi, Neta Kenneth, Veronika Mitnik, Lilja Maria Saeboe, Omri Abend
| Challenge: | Existing work on quantifying the prevalence of syntactic divergences across languages has not been done. |
| Approach: | They propose a framework for extracting divergence patterns for any language pair from a parallel corpus building on Universal Dependencies. |
| Outcome: | The proposed framework provides a detailed picture of cross-language divergences, generalizes previous approaches, and lends itself to full automation. |
SelectNoise: Unsupervised Noise Injection to Enable Zero-Shot Machine Translation for Extremely Low-resource Languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Currently, MT systems for low-resource languages lack parallel data and monolingual data. |
| Approach: | They propose an unsupervised approach to generate noisy HRLs training data by selective candidate extraction and noise injection. |
| Outcome: | The proposed model outperforms strong baselines on 12 ELRLs in a zero-shot setting . |
EnerGIZAr: Leveraging GIZA++ for Effective Tokenizer Initialization (2025.findings-acl)
Copied to clipboard
| Challenge: | Continual pre-training has long been considered the default strategy for adapting models to non-English languages, but struggles with initializing new embeddings, especially for non-Latin scripts. |
| Approach: | They propose a method that leverages statistical word alignment techniques to improve continual pre-training by leveraging word alignment matrix between source and target tokens. |
| Outcome: | The proposed method outperforms existing methods on key NLP tasks including POS tagging, Sentiment Analysis, NLI, and NER in Hindi, Basque, Arabic and Korean. |
FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing (2024.naacl-long)
Copied to clipboard
| Challenge: | Standard fine-tuning of language models suffers with generalization to distribution shifts. |
| Approach: | They propose to use Fisher Information to investigate scheduled unfreezing algorithms for adapter-based cross-lingual task transfer to improve generalization to distribution shifts. |
| Outcome: | The proposed method achieves an average of 2 points improvement over four datasets compared to standard fine-tuning and provides empirical evidence for a theory-based justification of the proposed method. |
Tracing the Roots of Facts in Multilingual Language Models: Independent, Shared, and Transferred Knowledge (2024.eacl-long)
Copied to clipboard
| Challenge: | Using low-resource languages, multilingual language models (ML-LMs) have been developed to transfer factual knowledge across languages. |
| Approach: | They ask how ML-LMs acquire and represent factual knowledge . they use a multilingual factual information probing dataset to investigate ML . |
| Outcome: | The findings highlight the challenge of maintaining consistency factual knowledge across languages. |
A Multi-dimensional Evaluation of Tokenizer-free Multilingual Pretrained Models (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent work on tokenizer-free models shows promising results in cross-lingual transfer . previous work focused on reporting accuracy on a limited set of tasks and data settings . |
| Approach: | They compare tokenizer-free and subword-based models using various dimensions . they find subword models are still the most practical choice in many settings . |
| Outcome: | The proposed model improves cross-lingual transfer and reduces engineering overhead. |
Structural Contrastive Pretraining for Cross-Lingual Comprehension (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train multilingual language models using pretraining tasks like mask language modeling have yielded promising results on a wide range of downstream tasks. |
| Approach: | They propose a new task to align the structural words in a parallel sentence, enhancing models’ ability to comprehend cross-lingual representations. |
| Outcome: | The proposed task improves model's ability to comprehend cross-lingual representations by increasing the frequency of negative pairings. |
Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual text classification require task-specific training data in high-resource sources . labeling cost, task characteristics, and privacy concerns can hinder the use of cross-linguistic training . |
| Approach: | They propose a dictionary-based heterogeneous graph (DHGNet) that uses bilingual dictionaries for task-independent word embeddings. |
| Outcome: | The proposed method outperforms pretrained models even though it does not access to large corpora. |
Speaking Multiple Languages Affects the Moral Bias of Language Models (2023.findings-acl)
Copied to clipboard
Katharina Haemmerl, Bjoern Deiseroth, Patrick Schramowski, Jindřich Libovický, Constantin Rothkopf, Alexander Fraser, Kristian Kersting
| Challenge: | Pre-trained multilingual language models are often better on English than other languages . however, they are trained on varying amounts of data for each language . |
| Approach: | They apply the MORALDIRECTION framework to multilingual models and analyse their results . they find that PMLMs encode differing moral biases, but these do not correspond to cultural differences or commonalities in human opinions. |
| Outcome: | The proposed model captures moral norms from English and imposes them on other languages. |
A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation (2024.findings-naacl)
Copied to clipboard
| Challenge: | Multilingual modelling can improve machine translation for low-resource languages, partly through shared subword representations. |
| Approach: | They propose to use subword regularisation to promote synergy and BPE to facilitate cross-lingual transfer. |
| Outcome: | The proposed methods promote synergy and prevent interference across different linguistic typologies. |
Isomorphic Transfer of Syntactic Structures in Cross-Lingual NLP (P18-1)
Copied to clipboard
| Challenge: | Using frameworks such as Universal Dependencies (UD) to transfer knowledge between languages can be challenging because of variation in syntactic structures. |
| Approach: | They propose a typologically driven method which reduces anisomorphism in UD treebanks by considering both morphological and structural properties. |
| Outcome: | The proposed method is effective for machine translation and cross-lingual sentence similarity. |
When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training (2026.eacl-long)
Copied to clipboard
| Challenge: | Recent studies have found that Large Language Models process multilingual inputs in shared concept spaces, thought to support generalization and cross-lingual transfer. |
| Approach: | They investigate the development of language-agnostic concept spaces during pretraining of EuroLLM using the causal interpretability method of activation patching. |
| Outcome: | The proposed model is language-agnostic and enables cross-lingual transfer . the model is able to process multilingual inputs, but lacks cross-linguistic alignment . |
MELA: Multilingual Evaluation of Linguistic Acceptability (2024.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks on linguistic acceptability have been used to evaluate language models' ability to distinguish between acceptable and unacceptable sentences. |
| Approach: | They present the largest benchmark to date on linguistic acceptability: MELA . they establish LLM baselines on this benchmark and investigate cross-lingual transfer in acceptability judgements with XLM-R. |
| Outcome: | The proposed model outperforms open-source models on cross-lingual transfer in acceptability judgements. |
Uncertainty-Aware Cross-Lingual Transfer with Pseudo Partial Labels (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to train pre-trained language models for zero-shot cross-lingual tasks are noisy and lack confidence. |
| Approach: | They propose an uncertainty-aware cross-lingual transfer framework with pseudo-partial-label to maximize the utilization of unlabeled data by reducing noise. |
| Outcome: | The proposed framework outperforms baselines on named entity recognition and natural language inference tasks on 40 languages. |
NeighXLM: Enhancing Cross-Lingual Transfer in Low-Resource Languages via Neighbor-Augmented Contrastive Pretraining (2025.findings-emnlp)
Copied to clipboard
| Challenge: | NeighXLM is a neighbor-augmented contrastive pretraining framework . it exploits intra-language semantic relationships captured during pretraining to construct high-quality positive pairs. |
| Approach: | They propose a neighbor-augmented contrastive pretraining framework that mines semantic neighbors from unlabeled corpora to enrich target-language supervision. |
| Outcome: | The proposed framework enriches target-language supervision by mining semantic neighbors from unlabeled corpora. |
Investigating Multilingual NMT Representations at Scale (D19-1)
Copied to clipboard
| Challenge: | Multilingual Neural Machine Translation models have shown success in transfer learning settings, but their mode of transfer remains elusive. |
| Approach: | They propose to use a representation similarity framework to compare multilingual representations using a SVCCA representation similar to the previous work. |
| Outcome: | The proposed model can be used to compare representations across languages and layers. |
How Many Languages Make Good Multilingual Instruction Tuning? A Case Study on BLOOM (2025.coling-main)
Copied to clipboard
| Challenge: | Many large language models (LLMs) support many languages, while others only support a few, e.g. the Llama series. |
| Approach: | They present a case study on BLOOM to understand three pertinent factors affecting performance: the number of languages, language exposure, and similarity between training and test languages. |
| Outcome: | The proposed model can be used to perform multilingual tasks on 1 to 52 languages. |
Learn to Cross-lingual Transfer with Meta Graph Learning Across Heterogeneous Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing mPLM-based methods focus on designing costly model pre-training while ignoring equally crucial downstream adaptation. |
| Approach: | They propose a meta graph learning method that extracts meta-knowledge from historical CLT experiences to learn to cross-lingual transfer. |
| Outcome: | The proposed method can learn to cross-lingual transfer by extracting meta-knowledge from historical CLT experiences (tasks) it can also capture intrinsic language relationships to explicitly guide cross-linguistic transfer. |
XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning (2020.emnlp-main)
Copied to clipboard
| Challenge: | XCOPA dataset provides a typologically diverse dataset for commonsense reasoning in 11 languages . current methods for evaluating commonsensible reasoning in resource-poor languages are weak compared to translation-based transfer. |
| Approach: | They propose a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages. |
| Outcome: | The proposed model performs better than current methods on a resource-poor dataset compared to translation-based transfer in the 11 languages studied . |
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT (2021.eacl-main)
Copied to clipboard
| Challenge: | Multilingual pretrained language models have demonstrated remarkable zero-shot cross-lingual transfer capabilities. |
| Approach: | They propose to use a layer ablation technique to create a multilingual model that is viewed as a stacking of two sub-networks: a language-agnostic encoder and a task-specific predictor. |
| Outcome: | The proposed model can perform zero-shot cross-lingual transfer for many languages. |
Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformers (2023.findings-acl)
Copied to clipboard
| Challenge: | despite lack of explicit cross-lingual training data, multilingual models can achieve cross-linguistic transfer. |
| Approach: | They find alignment is significantly correlated with cross-lingual transfer . they advocate for further research on realignment methods for smaller models . |
| Outcome: | The proposed method outperforms XLM-R Large in POS-tagging between English and Arabic by +15.8 accuracy. |
UniteD-SRL: A Unified Dataset for Span- and Dependency-Based Multilingual and Cross-Lingual Semantic Role Labeling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Multilingual and cross-lingual Semantic Role Labeling (SRL) has attracted increasing attention as multilingual text representation techniques have become more effective and widely available. |
| Approach: | They propose a benchmark for multilingual and cross-lingual, span- and dependency-based SRL that provides expert-curated parallel annotations using a common predicate-argument structure inventory. |
| Outcome: | The proposed benchmark provides expert-curated parallel annotations using a common predicate-argument structure inventory, allowing direct comparisons across languages and encouraging studies on cross-lingual transfer in SRL. |
Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) require vast datasets for pretraining, making it difficult to train LLMs from scratch for lowresource languages. |
| Approach: | They propose to transform a language of the GLUE benchmark and then fine tune a pretrained model on that dataset. |
| Outcome: | The proposed models recover from syntactic-style shifts, but cannot recover from vocabulary misalignment and embedding matrix re-initialization, even with continued pretraining on 15 million tokens. |
Dynamic Gazetteer Integration in Multilingual Models for Cross-Lingual and Cross-Domain Named Entity Recognition (2022.naacl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) models trained on CoNLL do not transfer well to other domains, even within the same language. |
| Approach: | They propose a token-level gating layer to augment pre-trained multilingual transformers with gazetteers containing named entities (NE) from a target language or domain. |
| Outcome: | The proposed model improves on cross-lingual transfer with an F1 score of 92.92 for English and an average of 89.43 across all languages in CoNLL. |
Struct-XLM: A Structure Discovery Multilingual Language Model for Enhancing Cross-lingual Transfer through Reinforcement Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods require syntactic labels that are difficult to obtain and of poor quality for low-resource languages. |
| Approach: | They propose a syntactic alignment model that leverages reinforcement learning to discover universal syntaktic structures for cross-lingual PLM alignment. |
| Outcome: | The proposed model improves cross-lingual representation alignment on the XTREME benchmark. |
Part-of-Speech Tagging on an Endangered Language: a Parallel Griko-Italian Resource (C18-1)
Copied to clipboard
Antonios Anastasopoulos, Marika Lekakou, Josep Quer, Eleni Zimianiti, Justin DeBenedetto, David Chiang
| Challenge: | a recent study examines POS tagging techniques on endangered languages . most natural language processing applications have been tested on only a handful of languages - a problem that is compounded by the lack of standard orthography. |
| Approach: | They evaluate POS tagging techniques on an endangered language, Griko . they use a semi-supervised method with cross-lingual transfer to achieve better accuracy . |
| Outcome: | The proposed method achieves 72.9% accuracy on a sample of 114 narratives in a language . the proposed method improves by 21 percentage points over previous methods . |
Cross-Lingual Word Embedding Refinement by ℓ1 Norm Optimisation (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for building high-quality CLWEs learn mappings that minimise the l2 norm loss function but this optimisation objective has been shown to be sensitive to outliers. |
| Approach: | They propose a simple post-processing step to improve cross-lingual word embeddings using the Manhattan norm goodness-of-fit criterion. |
| Outcome: | The proposed approach outperforms four state-of-the-art baselines in bilingual lexicon induction and cross-lingual transfer tasks. |
Scaling Laws for Multilingual Language Models (2025.findings-acl)
Copied to clipboard
Yifei He, Alon Benhaim, Barun Patra, Praneetha Vaddamanu, Sanchit Ahuja, Parul Chopra, Vishrav Chaudhary, Han Zhao, Xia Song
| Challenge: | Existing scaling laws for language models are limited to a limited number of languages, but they can be applied to arbitrary number of different languages. |
| Approach: | They propose a scaling law for general-purpose decoder-only language models trained on multilingual data that shifts focus from individual languages to language families. |
| Outcome: | The proposed scaling law can be applied to models trained on multilingual data . it can be used to predict performance across multiple languages and models . |
Project-then-Transfer: Effective Two-stage Cross-lingual Transfer for Semantic Dependency Parsing (2021.eacl-main)
Copied to clipboard
| Challenge: | Several remarkable contributions have been made in syntactic dependency parsing, especially on universal dependencies. |
| Approach: | They propose to capture cross-linguality by combing annotation projection and model transfer of pre-trained language models. |
| Outcome: | The proposed model parser almost achieved the approximated upper bound. |
Language Contamination Helps Explains the Cross-lingual Capabilities of English Pretrained Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | a large number of pretraining corpora are not publicly available, and it is unclear how much foreign language data exists in monolingual models. |
| Approach: | They propose to use English pretraining corpora to analyze their language composition . they find that even when less than 1% of data is not English, it facilitates cross-lingual transfer . |
| Outcome: | The proposed model is not truly monolingual when pretrained at scale, the authors show . they show that even when less than 1% of data is not English, it facilitates cross-lingual transfer . |
Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on multilingual models have focused on their cross-lingual transfer behavior . a recent study examined multilingual model learning from the multilingual pretraining signal . |
| Approach: | They analyze checkpoints during multilingual pretraining to identify when models acquire in-language and cross-lingual abilities. |
| Outcome: | The proposed model achieves high in-language performance early on, with lower-level linguistic skills acquired before more complex ones. |
Unifying Cross-Lingual Transfer across Scenarios of Resource Scarcity (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to deal with resource scarcity have not been developed to deal effectively with the problem. |
| Approach: | They propose to use a set of tools to harness data from one or more high-resource "source" languages to compensate for a shortage of data in low-resourced "target" languages. |
| Outcome: | The proposed technique can be easily adapted to unseen languages, extending the range of the proposed technique and translation-based transfer more broadly. |
Fine-tuning Encoders for Improved Monolingual and Zero-shot Polylingual Neural Topic Modeling (2021.naacl-main)
Copied to clipboard
| Challenge: | Topic models can augment or replace bag-of-words inputs with pre-trained transformer-based word prediction models. |
| Approach: | They propose several methods for fine-tuning encoders to improve both monolingual and zero-shot polylingual topic modeling. |
| Outcome: | The proposed methods improve both monolingual and zero-shot polylingual topic modeling. |
ZmBART: An Unsupervised Cross-lingual Transfer Framework for Language Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on large annotated training data. |
| Approach: | They propose an unsupervised framework that does not use parallel or pseudo-parallel/back-translated data. |
| Outcome: | The proposed framework does not use parallel or pseudo-parallel/back-translated data. |
MergeDistill: Merging Language Models using Pre-trained Distillation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained multilingual language models often lack capacity and skewed data . this leads to inequitable representation of languages due to limited capacity and sub-optimal vocabularies. |
| Approach: | They propose a framework to merge pre-trained multilingual language models to maximize their assets with minimal dependencies. |
| Outcome: | The proposed framework outperforms teacher-trained models on multiple datasets and with a fixed model capacity. |
Test-Time Code-Switching for Cross-lingual Aspect Sentiment Triplet Extraction (2025.naacl-long)
Copied to clipboard
| Challenge: | Aspect Sentiment Triplet Extraction (ASTE) is a thriving research area . current code-switching methods suffer from term boundary detection issues and out-of-dictionary problems. |
| Approach: | They propose a test-time code-switching framework which bridges the gap between bilingual training and monolingual test- time prediction. |
| Outcome: | The proposed framework achieves an average improvement of 3.7% on four cross-lingual datasets. |
Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer (2020.acl-main)
Copied to clipboard
| Challenge: | Multilingual word embeddings embed words from many languages into a single semantic space such that words with similar meanings are close to each other regardless of the language. |
| Approach: | They propose to use multilingual word embeddings to align embeddable words from multiple languages into a single semantic space so that words with similar meanings are close to each other regardless of the language. |
| Outcome: | The proposed model can be used to learn gender bias in multilingual representations and to improve transfer learning. |
Tracing Multilingual Knowledge Acquisition Dynamics in Domain Adaptation: A Case Study of Biomedical Adaptation (2026.eacl-long)
Copied to clipboard
| Challenge: | Multilingual domain adaptation (ML-DA) enables large language models to acquire domain knowledge across languages. |
| Approach: | They propose an adaptive evaluation method that constructs multiple-choice QA datasets from the same bilingual domain corpus used for training. |
| Outcome: | The proposed method constructs multiple-choice QA datasets from the same bilingual domain corpus used for training, thereby enabling direct analysis of multilingual knowledge acquisition. |
Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog (2022.naacl-main)
Copied to clipboard
| Challenge: | Task-oriented dialog (TOD) is arguably one of the most popular natural language processing (NLP) application areas. |
| Approach: | They propose a multilingual multi-domain TOD dataset that spans four languages . they use a framework for multilingual conversational specialization of pretrained language models . |
| Outcome: | The proposed datasets show that they perform better than existing datasets in English . the proposed framework allows for sample-efficient few-shot transfer for TOD tasks . |
Multilingual Content Moderation: A Case Study on Reddit (2023.eacl-main)
Copied to clipboard
| Challenge: | a growing need for AI moderators to safeguard users and protect mental health of human moderator from traumatic content. |
| Approach: | They propose to use a multilingual dataset to study the challenges of content moderation . they propose to analyze 1.8 million Reddit comments in English, german, spanish and french . |
| Outcome: | The proposed dataset highlights the challenges and suggests related research problems . it shows that the proposed model can be used to predict the violated rule . |
Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots (2021.naacl-main)
Copied to clipboard
| Challenge: | Multilingual models have demonstrated impressive cross-lingual transfer abilities. |
| Approach: | They propose two strong adversarial attacks that target multilingual models that can handle code-mixed sentences using bilingual dictionaries. |
| Outcome: | The proposed model has a success rate of 89.75% against XLM-R-large, bringing its average accuracy of 79.85 down to 8.18 on XNLI. |
X-METRA-ADA: Cross-lingual Meta-Transfer learning Adaptation to Natural Language Understanding and Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Multilingual models have gained popularity for their zero-shot cross-lingual transfer learning capabilities, but their generalization ability is inconsistent for typologically diverse languages. |
| Approach: | They propose a meta-learning approach that adapts MAML to learn to adapt to new languages . they extensively evaluate two cross-lingual NLU tasks using English as source and spanish as target . |
| Outcome: | The proposed approach outperforms naive fine-tuning on cross-lingual tasks for most languages. |
On the Calibration of Massively Multilingual Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Massively Multilingual Language Models (MMLMs) have gained popularity due to their effectiveness in cross-lingual transfer. |
| Approach: | They investigate how well calibrated MMLMs are with respect to confidence . they find that calibration methods like temperature scaling and label smoothing improve calibration . |
| Outcome: | The proposed models are able to generalize in languages unseen during fine-tuning, but they are not reliable across languages. |
ColBERT-XM: A Modular Multi-Vector Representation Model for Zero-Shot Multilingual Information Retrieval (2025.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to improve retrieval effectiveness in high-resource languages are limited due to the lack of high-quality labeled data in non-English languages. |
| Approach: | They propose a modular dense retrieval model that learns from the rich data of a single high-resource language and effectively zero-shot transfers to a wide array of languages. |
| Outcome: | The proposed model performs well against state-of-the-art multilingual retrieval models trained on more extensive datasets in various languages. |
To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages (2024.naacl-long)
Copied to clipboard
| Challenge: | XLT with multilingual language models is superfluous, says a new study . mBERT, XLM-R and mT5 are effective for cross-lingual transfer, authors say . |
| Approach: | They propose to use multilingual language models to improve cross-lingual transfer (XLT) they propose to add reliable translations to training data for XLT even for non-MT languages . |
| Outcome: | The proposed approaches outperform zero-shot XLT with mLMs, the authors show . the authors believe their findings warrant a broader inclusion of more robust translation-based baselines in XL research. |
MasakhaNER 2.0: Africa-centric Transfer Learning for Named Entity Recognition (2022.emnlp-main)
Copied to clipboard
David Adelani, Graham Neubig, Sebastian Ruder, Shruti Rijhwani, Michael Beukman, Chester Palen-Michel, Constantine Lignos, Jesujoba Alabi, Shamsuddeen Muhammad, Peter Nabende, Cheikh M. Bamba Dione, Andiswa Bukula, Rooweither Mabuya, Bonaventure F. P. Dossou, Blessing Sibanda, Happy Buzaaba, Jonathan Mukiibi, Godson Kalipe, Derguene Mbaye, Amelia Taylor, Fatoumata Kabore, Chris Chinenye Emezue, Anuoluwapo Aremu, Perez Ogayo, Catherine Gitau, Edwin Munkoh-Buabeng, Victoire Memdjokam Koagne, Allahsera Auguste Tapo, Tebogo Macucwa, Vukosi Marivate, Mboning Tchiaze Elvis, Tajuddeen Gwadabe, Tosin Adewumi, Orevaoghene Ahia, Joyce Nakatumba-Nabende, Neo Lerato Mokono, Ignatius Ezeani, Chiamaka Chukwuneke, Mofetoluwa Oluwaseun Adeyemi, Gilles Quentin Hacheme, Idris Abdulmumin, Odunayo Ogundepo, Oreen Yousuf, Tatiana Moteu, Dietrich Klakow
| Challenge: | Existing studies on named entity recognition methods for African languages focus on English as the source language, but there is evidence that it is not the best for low-resource languages. |
| Approach: | They propose to use human-annotated datasets to analyze named entity recognition tasks in 20 African languages to test whether they are effective. |
| Outcome: | The proposed method improves zero-shot F1 scores by 14% over 20 languages compared to using English . |
MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies on knowledge editing focus on monolingual scenarios, neglecting the complexities presented by multilingual contexts and multi-hop reasoning. |
| Approach: | They propose a benchmark to evaluate the adaptability of multilingual knowledge editing methods. |
| Outcome: | The proposed benchmark evaluates the adaptability of multilingual knowledge editing methods across five languages. |
Choosing Transfer Languages for Cross-Lingual Learning (P19-1)
Copied to clipboard
Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, Graham Neubig
| Challenge: | Cross-lingual transfer is a useful tool for improving performance of natural language processing (NLP) on low-resource languages. |
| Approach: | They propose to use cross-lingual transfer to improve accuracy of low-resource languages . they build models that consider features to perform prediction on such languages based on ranking problem . |
| Outcome: | The proposed model predicts good transfer languages much better than baselines considering single features in isolation. |
Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections (P19-1)
Copied to clipboard
| Challenge: | Current systems for syntactic analysis tasks rely heavily on large scale annotated data. |
| Approach: | They propose to learn a generative model with a structured prior that uses labeled source and unlabeled target data jointly. |
| Outcome: | The proposed model improves on part-of-speech tagging and dependency parsing tasks on English as the only source corpus and on a wide range of target languages. |
Multilingual Generative Language Models for Zero-Shot Cross-Lingual Event Argument Extraction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models for zero-shot cross-lingual event argument extraction are based on pre-trained generative language models. |
| Approach: | They propose to use pre-trained generative language models to generate sentences that fill in a template with arguments extracted from the input passage. |
| Outcome: | The proposed model outperforms the current state-of-the-art models on zero-shot cross-lingual EAE. |
Contextual Label Projection for Cross-Lingual Structured Prediction (2024.naacl-long)
Copied to clipboard
| Challenge: | Prior work favors simplified label translation or relying on word-level alignments for label projection. |
| Approach: | They propose a novel approach CLaP which translates text to target language and performs *contextual translation* on the labels using the translated text as the context. |
| Outcome: | The proposed approach improves translation accuracy on two prediction tasks and shows 2.4 F1 improvement for EAE and 1.4 F1 for named entity recognition. |
Improving Zero-Shot Cross-lingual Transfer Between Closely Related Languages by Injecting Character-Level Noise (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve cross-lingual transfer do not take surface similarity into account. |
| Approach: | They propose to augment source language training data with character-level noise to simulate spelling variations. |
| Outcome: | The proposed strategy shows consistent improvements over several languages and tasks. |
Blessing of Multilinguality: A Systematic Analysis of Multilingual In-Context Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a widely adopted technique for learning large language models . however, there is little systematic understanding of when and why it works well . |
| Approach: | They analyze multilingual in-context learning using demonstrations in HRLs to enhance cross-lingual transfer. |
| Outcome: | The proposed method outperforms English-only models on high-resource languages . the study shows that the presence of irrelevant non-English sentences in the prompt yields measurable gains . |
The interplay between language similarity and script on a novel multi-layer Algerian dialect corpus (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on cross-lingual transfer between languages with similar typology and languages of different scripts. |
| Approach: | They propose to annotate Algerian user-generated comments with parallel annotations . they also investigate the effect of script vs. language similarity in cross-lingual transfer . |
| Outcome: | The proposed model fine-tunes multi-lingual models on Algerian language and scripts . it shows that script vs. language similarity is important for part-of-speech tagging and sentiment analysis . |
Unifying Cross-Lingual and Cross-Modal Modeling Towards Weakly Supervised Multilingual Vision-Language Pre-training (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies address the problem of translating English data into other languages, but they are limited in form and scale. |
| Approach: | They propose a framework to unify cross-lingual and cross-modal pre-training by using English data. |
| Outcome: | The proposed framework unifies cross-lingual and cross-modal pre-training on different data. |
Investigating Transfer Learning in Multilingual Pre-trained Language Models through Chinese Natural Language Inference (2021.findings-acl)
Copied to clipboard
| Challenge: | Multilingual transformers have been shown to have remarkable transfer skills in zero-shot settings. |
| Approach: | They investigate cross-lingual transfer abilities of XLM-R for Chinese and English natural language inference using a large scale Chinese dataset. |
| Outcome: | The proposed model trains on Chinese and English natural language inference datasets. |
InteMATs: Integrating Granularity-Specific Multilingual Adapters for Cross-Lingual Transfer (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work relies on full-model fine-tuning on large parallel datasets to enhance cross-lingual alignment of MLLMs. |
| Approach: | They propose an approach that integrates multilingual adapters trained on texts of different levels of granularity into multilingual models. |
| Outcome: | The proposed approach improves the performance of multilingual language models on low-resource languages. |
Cross-lingual Multimodal Sentiment Analysis for Low-Resource Languages via Language Family Disentanglement and Rethinking Transfer (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing multimodal sentiment analysis methods are limited to textual data and cannot handle multimodal scenarios. |
| Approach: | They propose a transfer learning framework that allows cross-lingual and cross-modal alignments and a language family disentanglement module that enhances the sharing of language universals within families. |
| Outcome: | The proposed method is superior to existing methods and can handle low-resource languages. |
Cross-lingual Transfer Can Worsen Bias in Sentiment Analysis (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evidence of demographic bias in SA systems is limited to a handful of languages, and it is costly to create supervised training data in a new language. |
| Approach: | They use counterfactual evaluation to test whether gender or racial biases are imported when using cross-lingual transfer . r&r is much more prevalent than gender biase . |
| Outcome: | The proposed model is compared with monolingual systems in five languages and shows that it is biased more than monolingual ones. |
Syntax-augmented Multilingual BERT for Cross-lingual Transfer (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained multilingual text encoders capture language syntax, helping cross-lingual transfer. |
| Approach: | They provide language syntax and train mBERT to encode universal dependency tree structure. |
| Outcome: | The proposed model improves cross-lingual transfer on PAWS-X and MLQA benchmarks by 1.4 and 1.6 points on average across all languages. |
How to Adapt Your Pretrained Multilingual Model to 1600 Languages (2021.acl-long)
Copied to clipboard
| Challenge: | Pretrained multilingual models perform best for languages seen during pretraining . methods exist to improve performance for unseen languages, but have been evaluated using amounts of raw text only available for a small fraction of the world’s languages. |
| Approach: | They evaluate the performance of existing methods to adapt pretrained multilingual models to new languages using a resource available for close to 1600 languages: the New Testament. |
| Outcome: | The proposed models perform best for languages seen during pretraining . the results show that the most efficient approach is simplest and the most accurate . |
CORI: CJKV Benchmark with Romanization Integration - a Step towards Cross-lingual Transfer beyond Textual Scripts (2024.lrec-main)
Copied to clipboard
| Challenge: | Naively assuming English as a source language may hinder cross-lingual transfer . despite recent advances in cross-linguistic research, most studies have restricted themselves to two major assumptions . |
| Approach: | They propose to integrate Romanized transcription beyond textual scripts to capture contact between these languages . they propose to use a benchmark dataset to further encourage in-depth studies of language contact . |
| Outcome: | The proposed method allows for enhanced cross-lingual representations and effective zero-shot cross-linguistic transfer. |
Frustratingly Easy Label Projection for Cross-lingual Transfer (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve cross-lingual transfer performance are based on word alignment, but no empirical studies have evaluated their effectiveness or limitations. |
| Approach: | They propose a mark-then-translate method that integrates translation and projection by inserting special markers around the labeled spans in the original sentence. |
| Outcome: | The proposed method outperforms word alignment-based methods in 57 languages and three tasks. |
Cross-lingual Transfer or Machine Translation? On Data Augmentation for Monolingual Semantic Textual Similarity (2024.lrec-main)
Copied to clipboard
| Challenge: | Using labeled NLI datasets for learning sentence embeddings leads to improved performance for natural language understanding tasks. |
| Approach: | They compare two data augmentation techniques for learning better sentence embeddings . they use a cross-lingual transfer technique that exploits English resources as training data to yield non-English sentence embeds as zero-shot inference . |
| Outcome: | The proposed techniques yield better performance on Japanese and Korean sentences. |
Cross-Lingual Transfer Robustness to Lower-Resource Languages on Adversarial Datasets (2024.lrec-main)
Copied to clipboard
| Challenge: | Multilingual Language Models (MLLMs) exhibit robust cross-lingual transfer capabilities for downstream tasks such as Named Entity Recognition (NER) challenges persist in MLLM implementations that are not cross-linguistically robust. |
| Approach: | They evaluate two well-known MLLMs on 13 pairs of languages with a geographic, genetic, or borrowing relationship. |
| Outcome: | The proposed models show that they can leverage information acquired in a source language and apply it to a target language. |
Does Meta-learning Help mBERT for Few-shot Question Generation in a Cross-lingual Transfer Setting for Indic Languages? (2022.coling-1)
Copied to clipboard
Aniruddha Roy, Rupak Kumar Thakur, Isha Sharma, Ashim Gupta, Amrith Krishna, Sudeshna Sarkar, Pawan Goyal
| Challenge: | Existing approaches to few-shot Question Generation (QG) are limited and require manual annotation. |
| Approach: | They propose to use multilingual BERT to perform few-shot question generation with cross-lingual transfer. |
| Outcome: | The proposed model improves in few-shot QG and human evaluation confirms it. |
Neuron Specialization: Leveraging Intrinsic Task Modularity for Multilingual Machine Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Language-specific modeling methods that focus on heuristics for allocation of capacity and lack knowledge transfer capabilities are often prone to interference due to conflicting optimization demands. |
| Approach: | They propose a method that identifies specialized neurons to modularize feed-forward layers and updates them through sparse networks to avoid interference under multilingual translation. |
| Outcome: | The proposed approach achieves consistent performance gains over strong baselines with additional analyses showing reduced interference and increased knowledge transfer. |
Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning (2022.coling-1)
Copied to clipboard
| Challenge: | Multilingual pre-trained language models have shown impressive performance on several downstream tasks for both high-resourced and low-resource languages. |
| Approach: | They propose to apply multilingual adaptive fine-tuning to 17 most-resourced African languages and three other high-resource languages to encourage cross-lingual transfer learning. |
| Outcome: | The proposed approach is competitive to LAFT on individual languages while requiring significantly less disk space. |
Translation and Fusion Improves Cross-lingual Information Extraction (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown significant progress in information extraction tasks due to lack of labeled data for fine-tuning and unlabeled text for pre-training. |
| Approach: | They propose a framework in which large language models are fine-tuned to use English translations of low-resource language data. |
| Outcome: | The proposed model improves cross-lingual transfer over the base model on 12 multilingual IE datasets spanning 50 languages. |
Cross-lingual Multi-Level Adversarial Transfer to Enhance Low-Resource Name Tagging (N19-1)
Copied to clipboard
| Challenge: | Low-resource language name tagging is an important but challenging task. |
| Approach: | They propose a neural architecture that leverages multi-level adversarial transfer to improve name tagging for low-resource languages. |
| Outcome: | The proposed approach outperforms previous approaches on CoNLL data sets. |
Soft Language Clustering for Multilingual Model Pre-training (2023.acl-long)
Copied to clipboard
| Challenge: | Multilingual pre-trained language models have demonstrated impressive (zero-shot) cross-lingual transfer abilities, however, their performance is hindered when the target language has distant typology from the source language or when pre-training data is limited in size. |
| Approach: | They propose a method that contextually retrieves prompts as flexible guidance for encoding instances conditionally. |
| Outcome: | The proposed method improves on the XTREME task and also for low-resource languages in unsupervised sentence retrieval. |
JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset (2022.findings-emnlp)
Copied to clipboard
| Challenge: | JamPatoisNLI provides the first dataset for natural language inference in a creole language, Jamaican Patois. |
| Approach: | They propose to use JamPatoisNLI to provide a dataset for natural language inference in a creole language, Jamaican Patois. |
| Outcome: | The proposed dataset shows that the language's relatedness to English significantly boosts the effectiveness of cross-lingual transfer even for the higher order task. |
Applying Natural Annotation and Curriculum Learning to Named Entity Recognition for Under-Resourced Languages (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to build NLP models for low-resourced languages rely on machine translation or cross-lingual transfer. |
| Approach: | They propose to use natural annotations to build synthetic training sets from resources not originally designed for the target downstream task. |
| Outcome: | The proposed model achieves the F1 score of 0.78 for Belarusian starting from zero resources compared to the baseline of 0.63 for English . the proposed model can be fine-tuned to reflect linguistic properties, such as the grammatical case and gender, for the Slavic languages. |
Self-Distillation for Model Stacking Unlocks Cross-Lingual NLU in 200+ Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel on English NLU tasks, yet struggle to extend their NLU capabilities to underrepresented languages. |
| Approach: | They integrate machine translation models (MT) directly into LLM backbones via sample-efficient self-distillation. |
| Outcome: | The proposed model outperforms translation-test models on 127 low-resource languages. |
Functional Lexicon in Subword Tokenization (2025.naacl-long)
Copied to clipboard
| Challenge: | Function units are hard to map across languages, while being the most frequent tokens. |
| Approach: | They analyze subword tokens in terms of their productivity and try to find thresholds that best distinguish function from content tokens. |
| Outcome: | The proposed method can be used to identify functional lexical units in low-resource languages with minimal annotated data. |
Revisiting Machine Translation for Cross-lingual Classification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work in cross-lingual learning has pivoted around multilingual models, which are typically pretrained on unlabeled corpora in multiple languages using some form of language modeling objective. |
| Approach: | They propose to use a stronger machine translation system to mitigat mismatch between training on original text and running inference on machine translated text. |
| Outcome: | The proposed approach is highly task dependent and calls into question the dominance of multilingual models for cross-lingual classification. |
Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks. |
| Approach: | They do cross-lingual evaluation using prompt tuning and compare it with fine-tuning . prompt tuning achieves much better cross-linguistic transfer than fine- tuning . |
| Outcome: | The results show that prompt tuning achieves better cross-lingual transfer than fine-tuning across datasets, with only 0.1% to 0.3% tuned parameters. |
Code-Switching Curriculum Learning for Multilingual Transfer in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit near human-level performance in various tasks, but performance drops after a handful of high-resource languages due to the imbalance in pre-training data. |
| Approach: | They propose a code-switching curriculum learning model to enhance cross-lingual transfer for LLMs by progressively training models with a curriculum consisting of token-level code-changing, sentence-level codeswitching, and monolingual corpora. |
| Outcome: | The proposed model improves language transfer to Korean, with significant gains in Japanese and Indonesian . the proposed model mitigates spurious correlations between language resources and safety alignment . |
End-to-End Slot Alignment and Recognition for Cross-Lingual NLU (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to expand an NLU system to new languages use machine translation with slot label projection from source to translated utterances. |
| Approach: | They propose a novel end-to-end model that learns to align and predict target slot labels jointly for cross-lingual transfer. |
| Outcome: | The proposed model outperforms a simple label projection method on most languages and achieves competitive performance to the more complex, state-of-the-art projection method with only half the training time. |
LexFit: Lexical Fine-Tuning of Pretrained Language Models (2021.acl-long)
Copied to clipboard
| Challenge: | Transformer-based language models implicitly store a wealth of lexical semantic knowledge, but it is non-trivial to extract that knowledge effectively from their parameters. |
| Approach: | They propose to expose and enrich lexical knowledge from transformer-based language models to serve as effective decontextualized word encoders even when fed input words "in isolation" |
| Outcome: | The proposed model outperforms standard static WEs and vanilla LMs in lexical tasks over four established tasks in 8 languages. |
Towards Zero-Shot Multilingual Transfer for Code-Switched Responses (2023.acl-long)
Copied to clipboard
| Challenge: | Recent task-oriented dialog systems have had great success building English-based personal assistants, but extending these systems to a global audience may take tremendous efforts. |
| Approach: | They propose a framework that allows for efficient transfer by learning task-specific representations and encapsulating source and target language representations. |
| Outcome: | The proposed framework is able to successfully transfer language knowledge even when the target language corpus is limited. |
LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing language adaptation methods overlook the benefits of cross-lingual supervision. |
| Approach: | They propose a language adaptation tuning method that augments target corpus with English entity names and trains the model using left-to-right language modeling. |
| Outcome: | The proposed method shows significant performance gains across non-English languages. |
Automatic Extraction of Rules Governing Morphological Agreement (2020.emnlp-main)
Copied to clipboard
Aditi Chaudhary, Antonios Anastasopoulos, Adithya Pratapa, David R. Mortensen, Zaid Sheikh, Yulia Tsvetkov, Graham Neubig
| Challenge: | Creating a descriptive grammar is an indispensable step for language documentation but it is tedious and time-consuming. |
| Approach: | They propose a framework for extracting a first-pass grammatical specification from raw text in a concise, human- and machine-readable format. |
| Outcome: | The proposed framework extracts a grammatical specification that is nearly equivalent to those created with large amounts of gold-standard annotated data. |
ZGUL: Zero-shot Generalization to Unseen Languages using Multi-source Ensembling of Language Adapters (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to zero-shot cross-lingual transfer have focused on training with adapters of a single source and testing either with the target LA or LA of another related language. |
| Approach: | They propose to leverage LAs of multiple (linguistically or geographically related) source languages for more effective cross-lingual transfer instead of just one source LA . they extend their novel neural architecture, ZGUL, to settings where either (1) some unlabeled data or (2) few-shot training examples are available for the target language . |
| Outcome: | Extensive experimentation across four language groups, covering 15 unseen target languages, shows improvements of up to 3.2 average F1 points over baselines on POS tagging and NER tasks. |
AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages (2022.acl-long)
Copied to clipboard
Abteen Ebrahimi, Manuel Mager, Arturo Oncevay, Vishrav Chaudhary, Luis Chiruzzo, Angela Fan, John Ortega, Ricardo Ramos, Annette Rios, Ivan Vladimir Meza Ruiz, Gustavo Giménez-Lugo, Elisabeth Mager, Graham Neubig, Alexis Palmer, Rolando Coto-Solano, Thang Vu, Katharina Kann
| Challenge: | Pretrained multilingual models can perform cross-lingual transfer in a zero-shot setting, even for unseen languages. |
| Approach: | They propose to extend XNLI to 10 indigenous languages of the Americas and test multiple zero-shot and translation-based approaches. |
| Outcome: | The proposed model can perform cross-lingual transfer in a zero-shot setting even for languages unseen during pretraining. |
Candidate-Aware Retrieval and Reranking for Multiple-Choice Question Answering: Arabic as a Case Study (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have recently achieved impressive results on multiple-choice question answering (MCQA) despite advances in English, LLMs continue to underperform in Arabic due to gaps in data coverage, linguistic transfer, and evaluation design. |
| Approach: | They propose a method that jointly models the relevance of both the question and its candidate answers when selecting contextual passages. |
| Outcome: | The proposed approach outperforms standard RAG baselines and reranker baselines while remaining competitive with considerably larger models. |
MulDA: A Multilingual Data Augmentation Framework for Low-Resource Cross-Lingual NER (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual NER are labeled sequence translation and instance-based transfer via machine translation (MT) Existing methods to cross NER include label projection and labeling, but they are expensive and time-consuming. |
| Approach: | They propose a simple but effective labeled sequence translation method to translate source-language training data to target languages and avoids word order change and entity span determination. |
| Outcome: | The proposed method avoids word order change and entity span determination and can be generalized with the language-specific features from the target-language synthetic data and the language independent features from multilingual synthetic data. |
Analyzing the Effect of Linguistic Similarity on Cross-Lingual Transfer: Tasks and Experimental Setups Matter (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior work on cross-lingual transfer often focuses on a small set of languages from a few language families and/or a single task. |
| Approach: | They analyze cross-lingual transfer for 263 languages from a wide variety of language families . they include three popular NLP tasks: POS tagging, dependency parsing, topic classification . |
| Outcome: | The proposed approach is based on linguistic similarity measures for 263 languages . the results show that the effect of linguistic similarities on transfer performance depends on a range of factors . |
INJONGO: A Multicultural Intent Detection and Slot-filling Dataset for 16 African Languages (2025.acl-long)
Copied to clipboard
Hao Yu, Jesujoba Oluwadara Alabi, Andiswa Bukula, Jian Yun Zhuang, En-Shiun Annie Lee, Tadesse Kebede Guge, Israel Abebe Azime, Happy Buzaaba, Blessing Kudzaishe Sibanda, Godson Koffi Kalipe, Jonathan Mukiibi, Salomon Kabongo Kabenamualu, Mmasibidi Setaka, Lolwethu Ndolela, Nkiruka Odu, Rooweither Mabuya, Shamsuddeen Hassan Muhammad, Salomey Osei, Sokhar Samb, Dietrich Klakow, David Ifeoluwa Adelani
| Challenge: | Slot-filling and intent detection tasks are well-established tasks in Conversational AI, but current benchmarks for these tasks rely on evaluations of low-resource languages and translations from English benchmarks. |
| Approach: | They propose to use a multilingual, open-source benchmark dataset for 16 African languages with utterances generated by native speakers across diverse domains. |
| Outcome: | The proposed dataset compares multilingual transformer models and prompting large language models (LLMs) with the English language. |
Improving Zero-Shot Cross-lingual Transfer for Multilingual Question Answering over Knowledge Graph (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing approaches to solve multilingual question answering over knowledge graph (KGQA) use of bilingual lexicon induction to map training questions into those in target language circumvents language inconsistency . |
| Approach: | They propose to use bilingual lexicon induction to map training questions in source and target languages as augmented training data to minimize syntax-disorder. |
| Outcome: | The proposed model narrows the gap in zero-shot cross-lingual transfer between source and target languages. |
A Simple and Effective Method To Eliminate the Self Language Bias in Multilingual Representations (2021.emnlp-main)
Copied to clipboard
| Challenge: | Language agnostic and semantic-language information isolation is an emerging research direction for multilingual representations models. |
| Approach: | They propose a method that factors out language identity information from semantic related components in multilingual representations pre-trained on monolingual data. |
| Outcome: | The proposed method improves cross-lingual transfer performance on weak alignment models. |
Automatic Interlinear Glossing for Under-Resourced Languages Leveraging Translations (2020.coling-main)
Copied to clipboard
| Challenge: | Documentation is not a cure-all for language loss, but it is an important part of language preservation. |
| Approach: | They propose to use multi-source neural models to create automatic glossing models . they also explore cross-lingual transfer and a simple output length control mechanism . |
| Outcome: | The proposed model outperforms state-of-the-art models on low-resource scenarios. |
Multilingual Pre-training with Self-supervision from Global Co-occurrence Information (2023.findings-acl)
Copied to clipboard
| Challenge: | Empirical studies show multilinguality and crosslinguality emerge from MLM pretraining without supervision. |
| Approach: | They propose to use global co-occurrence information as a source of structural information on multilingual corpora. |
| Outcome: | Empirical studies show that MLM-GC pre-training outperforms MLM pre- training for 4 downstream cross-lingual tasks and 1 additional monolingual task. |
KINNEWS and KIRNEWS: Benchmarking Cross-Lingual Text Classification for Kinyarwanda and Kirundi (2020.coling-main)
Copied to clipboard
| Challenge: | low-resource African languages are traditionally left behind because of the lack of well-annotated data and effective preprocessing. |
| Approach: | They propose two news datasets for multi-class classification of news articles in two low-resource African languages. |
| Outcome: | The proposed datasets show that training embeddings on the higher-resourced Kinyarwanda yields successful cross-lingual transfer to Kirundi. |
Parameter-Efficient Cross-lingual Transfer of Vision and Language Models via Translation-based Alignment (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing cross-lingual transfer methods that use labeled data and linguistic resources would consume excessive resources for a large number of languages. |
| Approach: | They propose a parameter-efficient cross-lingual transfer learning framework that utilizes a translation-based alignment method to mitigate multilingual disparities. |
| Outcome: | The proposed framework reduces disparities among languages and improves cross-lingual transfer results in low-resource scenarios while keeping and fine-tuning only a small number of parameters. |
Cross-lingual Social Misinformation Detector based on Hierarchical Mixture-of-Experts Adapter (2025.coling-main)
Copied to clipboard
| Challenge: | a global trend of misinformation is affecting non-native speaker users who are more susceptible to misinformation on foreign social media platforms. |
| Approach: | They propose a method to integrate sentiment analysis as an auxiliary task and a hierarchical routing strategy and expert-mask mechanism to enhance cross-lingual social misinformation detection. |
| Outcome: | The proposed method improves cross-lingual social misinformation detection in non-native speakers with only monolingual social media histories. |
AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models excel in highresource languages but underperform in lowresource ones. |
| Approach: | They propose a cross-lingual transfer method that decouples "task ability" from "language ability" they propose to use adaptive adapter merging to obtain target adapters by combining other adapters. |
| Outcome: | The proposed method outperforms existing methods in highresource languages . it decouples "task ability" from "language ability" but fails to fully separate "task capability" from the "source language" |
On the Benefit of Syntactic Supervision for Cross-lingual Transfer in Semantic Role Labeling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in neural architectures and pre-trained representations have greatly improved the performance of fully-supervised semantic role labeling (SRL) but there are limitations in the availability of supervised training data. |
| Approach: | They propose to leverage syntactic dependencies to facilitate cross-lingual transfer by annotating predicate-argument structures in text. |
| Outcome: | The proposed model can be extended to other languages with limited training data. |
Subword Evenness (SuE) as a Predictor of Cross-lingual Transfer to Low-resource Languages (2022.emnlp-main)
Copied to clipboard
| Challenge: | English is the most natural choice for cross-lingual transfer, but it is often not the best choice for low-resource languages. |
| Approach: | They propose to use pre-trained multilingual models to improve performance in low-resource languages via cross-lingual transfer. |
| Outcome: | The results show that languages written in non-Latin and non-alphabetic scripts are the best choices for improving performance on Masked Language Modelling tasks in a diverse set of 30 low-resource languages. |
Discovering Language-neutral Sub-networks in Multilingual Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | a recent study shows that multilingual pre-trained language models transfer well on cross-lingual downstream tasks. |
| Approach: | They conceptualize language neutrality as a function of overlap between language-encoding sub-networks of multilingual models. |
| Outcome: | The proposed model performs well on cross-lingual tasks despite being pre-trained on multiple languages . |
WikiBank: Using Wikidata to Improve Multilingual Frame-Semantic Parsing (2020.lrec-1)
Copied to clipboard
| Challenge: | Frame-semantic annotations exist for a tiny fraction of the world’s languages, however, Wikidata provides a common, distant supervision signal for semantic parsers. |
| Approach: | They propose a multilingual resource with partial semantic dependency structures that can be used to extend pre-existing resources rather than creating new man-made resources from scratch. |
| Outcome: | The proposed resource can be used to augment pre-existing resources or reduce the annotation effort for low-resource languages. |
Distilling Efficient Language-Specific Models for Cross-Lingual Transfer (2023.findings-acl)
Copied to clipboard
| Challenge: | Massively multilingual Transformers (MMTs) are widely used for cross-lingual transfer learning. |
| Approach: | They propose to extract compressed, language-specific models from MMTs which retain the capacity of the original MMT for cross-lingual transfer. |
| Outcome: | The proposed model outperforms models trained from scratch in zero-shot cross-lingual transfer across benchmarks. |
Soft Language Prompts for Language Transfer (2025.naacl-long)
Copied to clipboard
| Challenge: | Cross-lingual knowledge transfer, especially between high- and low-resource languages, remains challenging in natural language processing. |
| Approach: | They propose to combine language-specific adapters and soft prompts to enhance cross-lingual transfer by parameter-efficient fine-tuning methods. |
| Outcome: | The proposed methods outperform language adapters and soft prompts in 16 languages and 10 low-resource languages. |
A Little Annotation does a Lot of Good: A Study in Bootstrapping Low-resource Named Entity Recognizers (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition models rely on large amounts of labeled data, making them challenging to extend to new, lower-resource languages. |
| Approach: | They propose a method for bootstrapping named entity recognition models in under-resourced languages . they use cross-lingual transfer learning and targeted annotation of only uncertain entities . |
| Outcome: | The proposed method achieves competitive accuracy with just one-tenth of training data. |
A Case Against Implicit Standards: Homophone Normalization in Machine Translation for Languages that use the Ge’ez Script. (2025.emnlp-main)
Copied to clipboard
Hellina Hailu Nigatu, Atnafu Lambebo Tonja, Henok Biadglign Ademtew, Hizkiel Mitiku Alemayehu, Negasi Haile Abadi, Tadesse Destaw Belay, Seid Muhie Yimam
| Challenge: | Homophone normalization is a pre-processing step used in Amharic natural language processing (NLP) but it also results in models that are unable to process different forms of writing in a single language. |
| Approach: | They propose a method where normalization is applied to model predictions instead of training data and a scheme where normalized data is preserved in training. |
| Outcome: | The proposed model achieves an increase in BLEU score of up to 1.03 while preserving language features in training. |
Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual pretrained language models (MPLMs) perform strongly in cross-lingual transfer. |
| Approach: | They propose to augment context with similar sentences retrieved from a high-resource language (HRL) they find a significant correlation between cross-lingual transfer performance and similarity between high- and low-resourced languages . |
| Outcome: | The proposed model outperforms finetuning by 3.7% on three downstream tasks with multilingual parallel test sets across 10 LRLs covering 6 language families in unlabeled and labeled settings. |
Weakly Supervised Cross-lingual Semantic Relation Classification via Knowledge Distillation (D19-1)
Copied to clipboard
| Challenge: | Existing work characterizes differences in meaning between words across languages using semantic relations . however, because of translation ambiguity, semantic relations are not always preserved by translation. |
| Approach: | They propose a cross-lingual relation classifier trained only with English examples and a bilingual dictionary to account for translation ambiguity when transferring knowledge from English to cross-linguistic settings. |
| Outcome: | The proposed model outperforms baselines that rely on bilingual embeddings or dictionaries for cross-lingual transfer and approaches fully supervised systems on English tasks. |
Emerging Cross-lingual Structure in Pretrained Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work has shown that multilingual pretraining works, but is unable to measure these effects. |
| Approach: | They propose to use multilingual masked language modeling to train a model on concatenated text from multiple languages to find universal latent symmetries in embedding spaces. |
| Outcome: | The proposed models can be trained on concatenated text from multiple languages without shared vocabulary or domain similarity. |
Romanization-based Large-scale Adaptation of Multilingual Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large multilingual pretrained language models are limited by their vocabulary size and parameter budget. |
| Approach: | They explore the potential of leveraging transliteration on a massive scale to improve performance for multilingual pretrained language models. |
| Outcome: | The proposed transliteration tool outperforms other methods on low-resource languages. |
A Survey of Multilingual Models for Automatic Speech Recognition (2022.lrec-1)
Copied to clipboard
| Challenge: | Automatic Speech Recognition (ASR) systems have achieved human-like performance for a few languages, but the majority of the world’s languages do not have usable systems due to the lack of large speech datasets to train these models. |
| Approach: | They propose to use unlabeled speech data to build multilingual ASR models that can be used for improved performance on low-resource languages. |
| Outcome: | The proposed models can be used to improve performance on low-resource languages by using unlabeled speech data. |
Isotropic Representation Can Improve Zero-Shot Cross-Lingual Transfer on Multilingual Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing mPLMs can align representations well for myriads of cross-lingual transfer tasks. |
| Approach: | They propose enhanced isotropy and constrained code-switching for zero-shot cross-lingual transfer to alleviate the problem of misalignment caused by anisotropic representations. |
| Outcome: | The proposed method improves on three zero-shot cross-lingual transfer tasks and over existing methods. |
Detecting and Mitigating Hallucinations in Multilingual Summarisation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing faithfulness metrics for abstractive summarisation models focus on English . metric mFACT is best suited to detect hallucinations in cross-lingual transfer . |
| Approach: | They propose a method to evaluate the faithfulness of non-English summaries by translation-based transfer from multiple English faithfulness metrics. |
| Outcome: | The proposed method reduces hallucinations in cross-lingual transfer by weighing the loss of each training example by its faithfulness score. |
Cross-Linguistic Syntactic Difference in Multilingual BERT: How Good is It and How Does It Affect Transfer? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual BERT (mBERT) has demonstrated considerable cross-lingual syntactic ability, but it is not well understood what leads to this variation and whether it fairly reflects difference between languages. |
| Approach: | They propose to use multilingual BERT to enable zero-shot cross-lingual transfer of syntactic knowledge between different languages by generating grammatical relations in 24 different languages. |
| Outcome: | The results show that the distance between the distributions of different languages is highly consistent with the syntactic difference in terms of linguistic formalisms. |
DaNE: A Named Entity Resource for Danish (2020.lrec-1)
Copied to clipboard
Rasmus Hvingelby, Amalie Brogaard Pauli, Maria Barrett, Christina Rosted, Lasse Malm Lidegaard, Anders Søgaard
| Challenge: | a named entity annotation for the Danish Universal Dependencies treebank is the largest publicly available named entity gold annotation. |
| Approach: | They propose a named entity annotation for the Danish Universal Dependencies treebank using the CoNLL-2003 annotation scheme DaNE. |
| Outcome: | The proposed annotations improve Danish named entity recognition over a recent cross-lingual approach and over norwegian training set. |
PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models exhibit reasonable multilingual abilities, despite predominantly English-centric pretraining. |
| Approach: | They propose a framework that establishes multilingual alignment prior to language model pretraining and preserves this alignment using a code-switching strategy during pretraining. |
| Outcome: | Experiments in a synthetic English to English-Clone setting show that PreAlign outperforms standard multilingual joint training in language modeling, zero-shot cross-lingual transfer, and cross-linguistic knowledge application. |
Enhancing Cross-lingual Transfer via Phonemic Transcription Integration (2023.findings-acl)
Copied to clipboard
| Challenge: | Previous cross-lingual transfer methods are limited to orthographic representation learning via textual scripts. |
| Approach: | They propose a phonemic transcription framework that incorporates phonemic translations as an additional linguistic modality beyond the orthographic transcriptions for cross-lingual transfer. |
| Outcome: | The proposed framework captures local one-to-one alignment between two different modalities and integrates bilingual dictionaries into multilingual contexts. |
Event Extraction in Basque: Typologically Motivated Cross-Lingual Transfer-Learning Analysis (2024.lrec-main)
Copied to clipboard
| Challenge: | Using a multilingual language model, Event Extraction tasks require humans to follow complicated guidelines and follow complicated rules. |
| Approach: | They propose a multilingual multilingual language model that is trained in a source language and applied to a target language. |
| Outcome: | The proposed model is based on a multilingual event extraction dataset for Basque . it shows that the shared linguistic characteristic between source and target languages does have an impact on transfer quality. |
Few-shot Learning with Multilingual Generative Language Models (2022.emnlp-main)
Copied to clipboard
Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyanov, Xian Li
| Challenge: | Large-scale generative language models such as GPT-3 are competitive few-shot learners. |
| Approach: | They train multilingual generative language models on a corpus covering a diverse set of languages and study their few- and zero-shot learning capabilities. |
| Outcome: | The proposed model outperforms GPT-3 on 171 out of 182 directions with 32 training examples and surpasses the official supervised baseline in 45 directions. |
Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | generative multilingual models fine-tuned on English forget to generate non-English data when labeled data is only available in English . generative models fine tuned on English fail to generate multilingual summarization tasks when labeling data is available in other languages . |
| Approach: | They propose to use prompt tuning to overcome catastrophic forgetting in a generative task in . they assume a strict setting with no parallel data or machine translation . |
| Outcome: | The proposed method can overcome catastrophic forgetting to enable zero-shot cross-lingual generation. |
Qwen2.5-xCoder: Multi-Agent Collaboration for Multilingual Code Instruction Tuning (2025.acl-long)
Copied to clipboard
Jian Yang, Wei Zhang, Yibo Miao, Shanghaoran Quan, Zhenhe Wu, Qiyao Peng, Liqun Yang, Tianyu Liu, Zeyu Cui, Binyuan Hui, Junyang Lin
| Challenge: | Existing methods to train code LLMs view each programming language in isolation . experimental results show that Qwen2.5-xCoder can bridge the gap between different programming languages . |
| Approach: | They propose a framework that allows agents to collaborate to enhance multilingual instruction tuning for code LLMs. |
| Outcome: | Experimental results show that Qwen2.5-xCoder can transfer knowledge efficiently and effectively between languages. |
The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm (2024.emnlp-main)
Copied to clipboard
null Aakanksha, Arash Ahmadian, Beyza Ermis, Seraphina Goldfarb-Tarrant, Julia Kreutzer, Marzieh Fadaee, Sara Hooker
| Challenge: | Existing approaches to safety alignment focus on homogeneous monolingual settings . preference training and safety measures often overfit to harms common in Western-centric datasets . |
| Approach: | They propose to use human annotated red teaming prompts to identify global and local harms. |
| Outcome: | The proposed approach can address and optimize for a non-homogeneous set of languages and cultural preferences while minimizing both global and local harms. |
Investigating Cultural Alignment of Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used to represent the diversity of human experience and culturally sensitive topics. |
| Approach: | They propose a method leveraging anthropological reasoning to enhance cultural alignment by prompting LLMs with different pretraining data mixtures in Arabic and English. |
| Outcome: | The proposed method enables users to better represent the diversity of human experience and the plurality of different cultures. |
What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition models are challenging for languages with little training data. |
| Approach: | They propose a simple and efficient neural architecture for cross-lingual named entity recognition models. |
| Outcome: | The proposed model achieves competitive performance with the state-of-the-art on two transferable factors: sequential order and multilingual embedding. |
mRAKL: Multilingual Retrieval-Augmented Knowledge Graph Construction for Low-Resourced Languages (2025.findings-acl)
Copied to clipboard
| Challenge: | Knowledge Graphs are structured multirelational graphs that store factual knowledge. |
| Approach: | They introduce a Retrieval-Augmented Generation (mRAKL) based system to perform mKGC. |
| Outcome: | The proposed approach improves over a no-context setting with an idealized retrieval system. |
How to Improve LLMs’ Performance on Specific Languages: A Perspective on LLM-Derived Language Similarity (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit uneven performance across languages. |
| Approach: | They propose to use a framework to quantify the similarity within each language pair through both the lenses of language-specific performance patterns and cross-lingual transferability. |
| Outcome: | The proposed approach outperforms traditional linguistic typology and cross-lingual transferability measures on multilingual LLMs. |
LiveCLKTBench: Towards Reliable Evaluation of Cross-Lingual Knowledge Transfer in Multilingual LLMs (2026.acl-long)
Copied to clipboard
Pei-Fu Guo, Yun-Da Tsai, Chun-Chia Hsu, Kai-Xin Chen, Ya An Tsai, Kai-Wei Chang, Nanyun Peng, Mi-Yen Yeh, Shou-De Lin
| Challenge: | Evaluating cross-lingual knowledge transfer in large language models is challenging, as correct answers in a target language may arise either from genuine transfer or from prior exposure during pre-training. |
| Approach: | They propose a pipeline to isolate and measure cross-lingual knowledge transfer by identifying self-contained, time-sensitive knowledge entities from real-world domains and generating factual questions. |
| Outcome: | The proposed pipeline analyzes multiple LLMs across five languages and shows that cross-lingual transfer is strongly influenced by linguistic distance and often asymmetric across language directions. |
Probing the Emergence of Cross-lingual Alignment during LLM Training (2024.findings-acl)
Copied to clipboard
| Challenge: | Multilingual Large Language Models (LLMs) achieve remarkable levels of zero-shot cross-lingual transfer performance. |
| Approach: | They propose that LLMs can align languages without explicit supervision from parallel sentences without a single linguistic feature. |
| Outcome: | The proposed model can perform zero-shot cross-lingual transfer even when the vocabularies of two languages have a null intersection, i.e., no tokens are shared. |
RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Preference optimization is a widely adopted post-training technique to align large language models with human preferences. |
| Approach: | They propose a method for generating multilingual feedback data to balance data coverage. |
| Outcome: | The proposed method achieves 54.4% win-rate against current state-of-the-art multilingual LLM in its parameter class and 69.5% win- rate or higher against widely used models like Gemma, Mistral and Llama 3. |
Cross-Lingual Optimization for Language Transfer in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Adapting large language models to other languages often suffers from an overemphasis on English performance. |
| Approach: | They propose a cross-lingual optimization technique that efficiently transfers an English-centric LLM to a target language while preserving its English capabilities. |
| Outcome: | The proposed model outperforms SFT in acquiring target language proficiency and maintaining English performance in low-resource languages. |
Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that target a single language are not seen during finetuning, but are able to respond in multiple languages once deployed in downstream applications. |
| Approach: | They investigate the minimal amount of multilinguality required during finetuning to elicit effective cross-lingual generalisation in English-centric LLMs. |
| Outcome: | The proposed model can respond in as few as two to three languages to a user's query in English, but the degree to which a target language is seen during pretraining is limiting. |
Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Effective cross-lingual transfer is hindered by performance gaps and the scarcity of fine-tuning data in many languages. |
| Approach: | They propose a middle-layer alignment objective integrated into task-specific training to improve cross-lingual transfer across languages. |
| Outcome: | The proposed method improves cross-lingual transfer to lower-resource languages and can be merged with existing modules without full re-training. |
Cross-lingual Back-Parsing: Utterance Synthesis from Meaning Representation for Zero-Resource Semantic Parsing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to extend semantic parsing (SP) beyond English are challenging due to the complex slot alignment step after translation. |
| Approach: | They propose a method to enhance cross-lingual transfer for SP by utilizing mPLMs. |
| Outcome: | The proposed method synthesizes target language utterances from source meaning representations while maintaining high slot value alignment rates. |
Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data? (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual transfer learning cannot activate cross-linguistic transferability when end-task data are unavailable. |
| Approach: | They propose a cross-lingual transfer method that disassembles multilingual Transformers into sub-modules and reassembles them to be the multilingual end-task model. |
| Outcome: | The proposed method activates the cross-lingual transferability of multilingual Transformers without accessing end-task data. |
UNKs Everywhere: Adapting Multilingual Language Models to New Scripts (2021.emnlp-main)
Copied to clipboard
| Challenge: | Massively multilingual language models offer state-of-the-art cross-lingual transfer performance on a range of NLP tasks, but there is a profound performance gap between resource-rich and resource-poor target languages. |
| Approach: | They propose a series of data-efficient methods that enable quick and effective adaptation of pretrained multilingual models to low-resource languages and unseen scripts. |
| Outcome: | The proposed methods improve learning of the new dedicated embedding matrix in the target language and for low-resource languages written in unseen scripts. |
Visually Grounded Reasoning across Languages and Cultures (2021.emnlp-main)
Copied to clipboard
| Challenge: | a new protocol allows for a multilingual hierarchy of concepts and images based on native speakers . the results suggest that the current models are not robust enough to handle multilingual data . |
| Approach: | They propose a protocol to construct an ImageNet-style hierarchy representative of more languages and cultures. |
| Outcome: | The proposed protocol lets the selection of concepts and images be entirely driven by native speakers, rather than scraping them automatically. |
Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that pixel representations can be finetuned across scripts without vocabulary extensions, adapters, or transliteration. |
| Approach: | They propose to use pixel representations to train multilingual machine translation models . they explore parameter sharing within and across scripts to better understand where they lead to positive transfer . |
| Outcome: | The proposed model improves on two multilingual datasets with different language coverage compared to subword embeddings . the proposed model can be finetuned cross-lingually or to unseen scripts, and is more data-efficient than other alternatives such as vocabulary expansion . |
Second Language Acquisition of Neural Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | a recent study examined the cross-lingual transferability of neural language models . previous studies focused on their first language acquisition . |
| Approach: | They propose to pretrain bilingual LMs with a scenario similar to human L2 acquisition . they find that pretraining accelerated their linguistic generalization in L2 . |
| Outcome: | The results show that pretraining bilingual LMs accelerates their linguistic generalizations . the results clarify their (non-)human-like L2 acquisition in particular aspects . |
From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | toxicity mitigation in language models has been focused on single-language settings . however, widespread adoption of LLMs has introduced a range of unknown -harms . |
| Approach: | They employ translated data to evaluate and enhance mitigation techniques in the absence of sufficient annotated datasets across languages. |
| Outcome: | The proposed approach compares translation quality and retrieval-augmented mitigation techniques under static and continual toxicity mitigation scenarios. |
Transitioning Representations between Languages for Cross-lingual Event Detection via Langevin Dynamics (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing datasets for event detection (ED) are limited to a small set of popular languages due to the high cost of data annotation. |
| Approach: | They propose a method to develop cross-lingual transfer learning models in high-resource source languages . they aim to transition the representations for target-language examples into the source-language space . |
| Outcome: | Extensive experiments over three languages demonstrate the state-of-the-art in CLTL. |
A Multi-Modal Multilingual Benchmark for Document Image Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing document image classification datasets have several limitations and we present two new datasets that overcome these limitations. |
| Approach: | They propose to use two newly curated multilingual datasets that overcome these limitations and propose to develop multilingual Document AI models. |
| Outcome: | The proposed datasets overcome limitations in document image classification and open the door for future research into improving Document AI models. |
IRIS: Interleaved Reinforcement with Incremental Staged Curriculum for Cross-Lingual Mathematical Reasoning (2026.acl-long)
Copied to clipboard
Navya Gupta, Rishitej Reddy Vyalla, Avinash Anand, Chhavi Kirtani, Erik Cambria, Zhengchen Zhang, Zhengkui Wang, Timothy Liu, Aik Beng Ng, Simon See, Rajiv Ratn Shah
| Challenge: | Curriculum learning fails to generate consistent step-by-step reasoning in multilingual and low-resource settings. |
| Approach: | They propose a framework that combines supervised fine-tuning with reverse curriculum reinforcement learning to generate consistent step-by-step reasoning. |
| Outcome: | The proposed framework outperforms single-axis benchmarks and multilingual test sets on math reasoning tasks and in high-resource languages. |
NGLUEni: Benchmarking and Adapting Pretrained Language Models for Nguni Languages (2024.lrec-main)
Copied to clipboard
| Challenge: | Nguni languages have over 20 million home language speakers in South Africa . there has been considerable growth in the datasets for these languages, but no analysis of the performance of NLP models for these language has been reported across languages and tasks. |
| Approach: | They compile publicly available datasets for natural language understanding and generation, spanning 6 tasks and 11 datasets. |
| Outcome: | The proposed models outperform existing models and large-scale adapted models on cross-lingual transfer and machine translation. |
A Reinforcement Learning Framework for Cross-Lingual Stance Detection Using Chain-of-Thought Alignment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual stance detection can't effectively perform cross-linguistic transfer of complex reasoning processes. |
| Approach: | They propose a framework to facilitate cross-lingual transfer of complex reasoning processes in stance detection by using cross-linguistic Chain-of-Thought alignment to obtain high-quality CoTs generated from target language inputs. |
| Outcome: | The proposed framework outperforms competing models on four multilingual datasets. |
Improving Occupational ISCO Classification of Multilingual Swiss Job Postings with LLM-Refined Training Data (2025.findings-acl)
Copied to clipboard
| Challenge: | 80% of job postings are German, 11% French, 8% English, and under 1% Italian. |
| Approach: | They propose a method that refines silver-standard ISCO labels by consolidating them with predictions from pre-fine-tuned models to resolve discrepancies. |
| Outcome: | The proposed method raises Top-1 accuracy on silver data to 58.3% and reaches 80% precision on held-out data. |
TransAlign: Machine Translation Encoders are Strong Word Aligners, Too (2025.findings-emnlp)
Copied to clipboard
| Challenge: | translation-based approaches to cross-lingual transfer (XLT) are limited. |
| Approach: | They propose a word aligner that utilizes the encoder of a massively multilingual MT model. |
| Outcome: | The proposed word aligner outperforms existing WA and state-of-the-art non-WA-based methods in token classification tasks. |
False Friends Are Not Foes: Investigating Vocabulary Overlap in Multilingual Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has shown that token overlap facilitates cross-lingual transfer or introduces interference between languages? |
| Approach: | They devised a controlled experiment where they train bilingual autoregressive models on multiple language pairs under systematically varied vocabulary overlap settings. |
| Outcome: | The proposed model outperforms models with disjointed vocabularies on XNLI and XQuAD and shows that token overlap is beneficial for multilingual tokenizers. |
NiuTrans.LMT: Toward Inclusive and Scalable Multilingual Machine Translation with LLMs (2026.acl-long)
Copied to clipboard
Yingfeng Luo, Ziqiang Xu, Yuxuan Ouyang, MuRun Yang, DingYang Lin, Kaiyan Chang, Tong Zheng, Bei Li, Peinan Feng, Quan Du, Tong Xiao, JingBo Zhu
| Challenge: | Large language models have significantly advanced Multilingual Machine Translation (MMT) yet scaling to many languages while maintaining robust performance across directions remains challenging. |
| Approach: | They propose a strategy to reduce the number of translations in one direction . they propose auxiliary parallel sentences to promote cross-lingual transfer . |
| Outcome: | The proposed model performs on par with or better than substantially larger baselines. |
No Culture Left Behind: ArtELingo-28, a Benchmark of WikiArt with Captions in 28 Languages (2024.emnlp-main)
Copied to clipboard
Youssef Mohamed, Runjia Li, Ibrahim Ahmad, Kilichbek Haydarov, Philip Torr, Kenneth Church, Mohamed Elhoseiny
| Challenge: | Traditionally, vision research focused on unambiguous class labels, whereas ArtELingo emphasizes diversity of opinions over languages and cultures. |
| Approach: | They propose a vision-language benchmark that spans 28 languages and encompasses approximately 200,000 annotations. |
| Outcome: | The proposed benchmark spans 28 languages and encompasses approximately 200,000 annotations . the challenge is to build machine learning systems that assign emotional captions to images . |
Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-training (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for enhancing cross-lingual transfer are limited by parallel resources and lack linguistic and domain coverage. |
| Approach: | They propose a cross-lingual in-context pre-training approach that leverages semantically related bilingual Wikipedia documents to enhance cross-linguistic transfer. |
| Outcome: | The proposed approach improves multilingual performance on three models across six target languages. |
You Are What You Train: Effects of Data Composition on Training Context-aware Machine Translation Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Using sparse contextually rich examples, we demonstrate a strong association between training data sparsity and model performance. |
| Approach: | They propose two training strategies to leverage contextually rich examples in training data . they demonstrate strong association between sparsity and model performance . |
| Outcome: | The proposed training strategies improve translation accuracy by 6 and 8 percentage points on the ctxPro evaluation. |
Towards Explainability and Fairness in Swiss Judgement Prediction: Benchmarking on a Multilingual Dataset (2024.lrec-main)
Copied to clipboard
| Challenge: | Using Swiss Judgement Prediction, we evaluate the explainability of state-of-the-art monolingual and multilingual LJP models. |
| Approach: | They propose an occlusion-based approach to evaluate the explainability performance of legal judgement prediction models using Swiss Judgement Prediction, the only available multilingual LJP dataset. |
| Outcome: | The proposed framework allows us to quantify the influence of lower court information on model predictions, exposing current models’ biases. |
Plutus: Benchmarking Large Language Models in Low-Resource Greek Finance (2025.emnlp-main)
Copied to clipboard
Xueqing Peng, Triantafillos Papadopoulos, Efstathia Soufleri, Polydoros Giannouris, Ruoyu Xiang, Yan Wang, Lingfei Qian, Jimin Huang, Qianqian Xie, Sophia Ananiadou
| Challenge: | Greek is the dominant language of the world's merchant navy and is a key language for international trade. |
| Approach: | They propose to develop a Greek financial evaluation benchmark and a financial LLM fine-tuned on Greek-specific financial data to bridge this gap. |
| Outcome: | The proposed benchmarks surpass GPT-4 by 8.33%, GPT- 4o by 26.83%, and Deepseek-V3 by 67.74%. |
Model-Based Ranking of Source Languages for Zero-Shot Cross-Lingual Transfer (2025.emnlp-main)
Copied to clipboard
| Challenge: | NN-Rank is an algorithm for ranking source languages for cross-lingual transfer . it leverages hidden representations from multilingual models and unlabeled target-language data . |
| Approach: | They propose an algorithm for ranking source languages for cross-lingual transfer which leverages hidden representations from multilingual models and unlabeled target-language data. |
| Outcome: | The proposed algorithm outperforms state-of-the-art models on in-domain data and shows that it can achieve 92.8% of the NDCG achieved using all available target data. |
UbuntuGuard: A Culturally-Grounded Policy Benchmark for Equitable AI Safety in African Languages. (2026.findings-acl)
Copied to clipboard
Tassallah Abdullahi, Macton Mgonzo, Mardiyyah Oduwole, Paul Okewunmi, Abraham Toluwase Owodunni, Ritambhara Singh, Carsten Eickhoff
| Challenge: | Current guardian models are predominantly Western-centric and optimized for high-resource languages . low-resourced African languages are vulnerable to evolving harms, cross-lingual failures, cultural misalignment . |
| Approach: | They propose a policy-based safety benchmark for African languages built from adversarial queries authored by 155 domain experts across sensitive fields. |
| Outcome: | The proposed model overestimates multilingual safety, cross-lingual transfer provides partial but insufficient coverage, and dynamic models struggle to localize African-language contexts. |
Just Use XML: Revisiting Joint Translation and Label Projection (2026.findings-acl)
Copied to clipboard
| Challenge: | Label projection is an effective technique for cross-lingual transfer, extending span-annotated datasets from high-resource languages to low-resourced ones. |
| Approach: | They propose a framework that performs translation and label projection via XML tags. |
| Outcome: | The proposed framework outperforms baselines and improves translation quality across languages and annotation complexity. |
One Script Instead of Hundreds? On Pretraining Romanized Encoder Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | a recent study has focused on setups that favor romanization for cross-lingual transfer . a fidelity-based approach is needed to improve performance for high-resource languages . |
| Approach: | They propose to pretrain LMs from scratch on romanized and original texts for six languages . they find that romanization improves encoding efficiency for segmental scripts at a negligible cost . |
| Outcome: | The proposed method reduces the loss of script-specific information and dilution of language-specific representations from increased subword overlap. |
Prosody as Supervision: Bridging the Non-Verbal–Verbal for Multilingual Speech Emotion Recognition (2026.acl-long)
Copied to clipboard
| Challenge: | Existing paradigms for low-resource multilingual speech emotion recognition rely on labeled verbal speech and lack cross-lingual transfer. |
| Approach: | They propose a paralinguistic supervision paradigm for low-resource multilingual speech emotion recognition that leverages non-verbal vocalizations to exploit prosody-centric emotion cues. |
| Outcome: | The proposed framework outperforms Euclidean counter parts and strong SSL baselines in the language-based evaluation of low-resource multilingual speech emotion recognition (LRM-SER) |